XSLT 3.0: xsl:package and xsl:use-package

It seemed to me that #127 is too broad, plus it's pretty old, so I wanted to pull this out and call a little more attention to it.  There are two points here:

XSpec won't run on xsl:package files.  Since the new package concept allows for reusable library code, people - I hope - would particularly want to write tests for XSL packages.
XSpec errors when running it on a stylesheet that contains xsl:use-package.  XSpec uses xsl:import to get the stylesheet under test, and using xsl:import is an error when trying to import a stylesheet which uses a package (see dependencies between packages in the spec).

I'm not sure how you get around the conflict between xsl:import and xsl:use-package, though, because I'm sure it's xsl:import and not xsl:include for a reason.
Package: using the example package here
<?xml version="1.0" encoding="UTF-8"?>
<xsl:package
    name="http://example.org/complex-arithmetic.xsl"
    package-version="1.0"
    version="3.0"
    xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:f="http://example.org/complex-arithmetic.xsl">

    <xsl:function name="f:complex-number"
        as="map(xs:integer, xs:double)" visibility="public">
        <xsl:param name="real" as="xs:double"/>
        <xsl:param name="imaginary" as="xs:double"/>
        <xsl:sequence select="map{ 0:$real, 1:$imaginary }"/>
    </xsl:function>

    <xsl:function name="f:real"
        as="xs:double" visibility="public">
        <xsl:param name="complex" as="map(xs:integer, xs:double)"/>
        <xsl:sequence select="$complex(0)"/>
    </xsl:function>

    <xsl:function name="f:imag"
        as="xs:double" visibility="public">
        <xsl:param name="complex" as="map(xs:integer, xs:double)"/>
        <xsl:sequence select="$complex(1)"/>
    </xsl:function>

    <xsl:function name="f:add"
        as="map(xs:integer, xs:double)" visibility="public">
        <xsl:param name="x" as="map(xs:integer, xs:double)"/>
        <xsl:param name="y" as="map(xs:integer, xs:double)"/>
        <xsl:sequence select="
            f:complex-number(
            f:real($x) + f:real($y),
            f:imag($x) + f:imag($y))"/>
    </xsl:function>

    <xsl:function name="f:multiply"
        as="map(xs:integer, xs:double)" visibility="public">
        <xsl:param name="x" as="map(xs:integer, xs:double)"/>
        <xsl:param name="y" as="map(xs:integer, xs:double)"/>
        <xsl:sequence select="
            f:complex-number(
            f:real($x)*f:real($y) - f:imag($x)*f:imag($y),
            f:real($x)*f:imag($y) + f:imag($x)*f:real($y))"/>
    </xsl:function>
</xsl:package>

Stylesheet with use-package:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:f="http://example.org/complex-arithmetic.xsl"
    exclude-result-prefixes="xs"
    version="3.0">
    <xsl:use-package name="http://example.org/complex-arithmetic.xsl" version="3.0"/>
    <xsl:template name="xsl:initial-template">
        <xsl:variable name="val1" select="f:complex-number(2,5)"/>
        <xsl:variable name="val2" select="f:complex-number(3,2)"/>
        <xsl:variable name="sum" select="f:add($val1, $val2)"/>
        <xsl:sequence expand-text="yes">The sum of the two numbers is {f:real($sum)} + {f:imag($sum)}i</xsl:sequence>
    </xsl:template>
</xsl:stylesheet>

XSpec test:
<?xml version="1.0" encoding="UTF-8"?>
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    stylesheet="use-package-example.xsl">
    <x:scenario label="Scenario for testing function initial-template">
        <x:call template="xsl:initial-template"/>
        <x:expect label="it returns the expected message" select="'The sum of the two numbers is 5 + 7i'"/>
    </x:scenario>
</x:description>

Here's the saxon config telling it how to find the package:
<?xml version="1.0" encoding="UTF-8"?>
<configuration xmlns="http://saxon.sf.net/ns/configuration">
    <xsltPackages>
        <package name="http://example.org/complex-arithmetic.xsl" version="3.0" sourceLocation="package-example.xsl" />
    </xsltPackages>
</configuration>

In Oxygen XML, everything shows up without errors, and the test stylesheet will run and produce output, but if you try to run the XSpec test, you will get:
run-xslt-test:
     [echo] Running XSLT Tests...
     [java] Error on line 9 column 115 of package-test-compiled.xsl:
     [java]   XTSE3008: xsl:use-package cannot appear in an imported stylesheet
     [java] xsl:use-package cannot appear in an imported stylesheet

I'm not sure how you get around the conflict between xsl:import and xsl:use-package, though, because I'm sure it's xsl:import and not xsl:include for a reason.

Without xsl:import, global x:param and x:variable cannot overwrite the ones in the stylesheet being tested.
So, with the given example, what if XSpec is written like this
<x:description stylesheet="package-example.xsl">
   <x:scenario>
      <x:call function="f:complex-number">
...

and it is compiled into like this?
<xsl:stylesheet>
   <xsl:use-package name="http://example.org/complex-arithmetic.xsl" version="1.0"/>
   <xsl:import href=".../src/compiler/generate-tests-utils.xsl"/>
...

Will it serve the majority of use cases?
Note that XSpec (CLI and Ant) already allows users to specify a Saxon configuration file to be used when running the compiled stylesheet.
Please be warned that I have zero experience with XSLT packages and am just looking at the given example :)

Thanks for taking the time to look at this.  :)

Without xsl:import, global x:param and x:variable cannot overwrite the ones in the stylesheet being tested.

Okay, that makes sense.  I've periodically been going through the source code to try and see how it works, but there's a lot in there to follow, and what XSpec does is completely unlike anything I've tried to do in XSLT.

So, with the given example, what if XSpec is written like this
<x:description stylesheet="package-example.xsl">
   <x:scenario>
      <x:call function="f:complex-number">
...

and it is compiled into like this?
<xsl:stylesheet>
   <xsl:use-package name="http://example.org/complex-arithmetic.xsl" version="1.0"/>
   <xsl:import href=".../src/compiler/generate-tests-utils.xsl"/>
...

Will it serve the majority of use cases?

If I'm understanding, this would allow me to test what's in the package itself (which is a good thing!) but I don't see a way that it would allow me to test any contents of the stylesheet that is using the package.  With such a simple set of examples, it's hard to come up with a convincing use case, but suppose we had something like:
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:f="http://example.org/complex-arithmetic.xsl"
    exclude-result-prefixes="xs"
    version="3.0">
    <xsl:use-package name="http://example.org/complex-arithmetic.xsl" version="3.0"/>
    <xsl:template match="MyNumbers">
        <xsl:variable name="real-numbers" as="xs:double*" select="Real/nbr"/>
        <xsl:variable name="imaginary-numbers" as="xs:double*" select="Imaginary/nbr"/>
        <xsl:complex-numbers select="for $r in $real-numbers, $i in $imaginary-numbers return f:complex-number($r, $1)"/>
        <xsl:sequence select="fold-left($complex-numbers, f:complex-number(0,0), function ($result, $input) { f:add($result, $input) })"/>
    </xsl:template>
</xsl:stylesheet>

So then I'd want to create an XSpec test for this (especially since I wrote this out without a reference and it could be full of syntax errors or have other problems) that verifies that something like:
<MyNumbers>
    <Real>
        <nbr>1</nbr>
        <nbr>4</nbr>
        <nbr>9</nbr>
    </Real>
    <Imaginary>
        <nbr>8</nbr>
        <nbr>2</nbr>
        <nbr>7</nbr>
    </Imaginary>
</MyNumbers>

when given as input to the template will yield the correct sum.
I'd absolutely be willing to try to add this functionality, but I'm not sure where I'd even begin, especially here - since you need the overrides from xsl:import, it's not obvious to me how to accomplish the same thing without running into the error of importing a stylesheet that uses a package.
Maybe as you're transitioning to a 3.0 codebase (I think I saw something about there being plans to make XSpec 2.0 based completely on XSLT 3.0?) there's a solution that uses packages in XSpec's compiler code...somehow?  Could the stylesheet under test maybe be recast as some sort of temporary package, allowing you to set visibility or override rules to get the same result as you can with xsl:import?  Probably not practical, but all I have are half-formed ideas and speculation at this point.

What if XSpec takes the following measures when it finds xsl:use-package in the stylesheet being tested?

Stop inserting global xsl:param (generated from global x:param) into the compiled stylesheet.
Stop writing <xsl:import href="stylesheet-to-be-tested.xsl" /> in the compiled stylesheet. Use xsl:include instead.
Do not run the compiled stylesheet directly. Instead, run it via fn:transform() with stylesheet-params option (generated from global x:param)

That sounds like a promising approach.

fn:transform() has its own limitations and uncertainty.
For now, what I come up with is a combination of these:

Implement xsl:use-package in XSpec
Include your main stylesheet from a stub main package where the named mode rules are redirected to the unnamed mode rules:<xsl:package declared-modes="no" exclude-result-prefixes="#all" name="my-main-package" version="3.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
   <xsl:include href="main-stylesheet.xsl" />
   <xsl:expose component="*" names="*" visibility="public" />
   <xsl:template match="document-node() | node() | attribute()" mode="my-mode">
      <xsl:apply-templates select="." />
   </xsl:template>
</xsl:package>

Test the stub main package from XSpec using a named mode:<x:description stylesheet="main-package.xsl" xmlns:x="http://www.jenitennison.com/xslt/xspec">
  <x:scenario label="using named mode">
     <x:context mode="my-mode" />
     <x:scenario label="test something">
        <!-- test.xml has <MyNumbers> -->
        <x:context href="test.xml" />
        <x:expect label="expect something" select="map{0:4.2e1,1:5.1e1}" />
     </x:scenario>
  </x:scenario>
</x:description>

Again, my knowledge of packaging is extremely limited. There may be a better way.

I think we could ask Michael Kay for help on this particular point. He previously helped XSpec project with the Coverage java code.
Did you commit on a particular branch, on which Mike could comment ?

No, I haven't made any commit. I just tested my previous comment with the given example.
What I probably need to know first (except for learning the packages itself...) would be a best practice when you have this working A.xsl
<!-- A.xsl -->
<xsl:stylesheet ...>
   <!-- Usual xsl:param, xsl:variable, xsl:function, xsl:template etc
      including unnamed mode xsl:template -->
</xsl:stylesheet>

and working B.xsl
<!-- B.xsl -->
<xsl:stylesheet ...>
   <xsl:import href="A.xsl" />
   <!-- Use stuff in A.xsl, possibly overriding some global xsl:param -->
</xsl:stylesheet>

and when xsl:use-package is introduced in A.xsl.
<!-- A.xsl revised -->
<xsl:stylesheet ...>
   <!-- NEW: Use package -->
   <xsl:use-package ... />

   <!-- The rest intact -->
</xsl:stylesheet>

What is the best practice to rewrite A.xsl and/or B.xsl so that both continue working?
Maybe the answer is "it depends"... But I can't go further, as I haven't used the packages myself. My previous comment just came up from Note ("If existing XSLT code has been written to use template rules in the unnamed mode...") in the spec.

I copy here mail exchanges with Michael Kay, on this topic :
Interesting. It's difficult to advise because I don't really know enough about the constraints imposed by the current architecture.
I'd be inclined to say: if the stylesheet under test uses packages, then it must itself be (or be treated as) a package, which means XSpec should reference it using xsl:use-package rather than using xsl:import.
But I'd also suggest that static linking of the stylesheet under test seems an unnatural thing to do when you've got fn:transform() to achieve dynamic invocation. Under Saxon-JS a lot of our test driver for the XSLT3 test suite is written in XSLT, and it invokes the stylesheet under test using fn:transform(). This seems cleaner because there's no risk of unintended overrides, plus you can test many different stylesheets in a single run with no conflicts of global variables etc. (But I guess XSpec is focussed on testing one stylesheet...)
Static variables and parameters are also worth considering here. Although it exposes a bit of a hole in the packaging spec, where it's rather unclear how xsl:use-package is supposed to supply values of static parameters in the used stylesheet.
Mike

On 28 Feb 2020, at 08:27, Christophe Marchand wrote:
Hello Mike,
sorry for this off-list message, but with XSpec guys, we need advices on how to implement unit tests on packages.
I've ask O'Neil and Debby last year at Prague, be we weren't ready to implement unit tests on packages - we had just discover that it'll be difficult...
Actually, when we test a XSLT, we generate a XSL that imports the one to test. It's ok, if the module to test is not a package.
To test packages, we do not know what is the best practise, to create a wrapper, to transform the package in a standard module, or another possibility.
Could you spend a small part of your spare time to give us some advices on the way to do it ? We have an issue open on github for this : https://github.com/xspec/xspec/issues/762

Thanks, @cmarchand
After experimenting xsl:include and xsl:use-package which are relatively trivial and work to some extent, I'm inclined toward fn:transform() for practical reasons. The biggest reason is that xsl:include and xsl:use-package benefit only the package users. Another reason is that we'll not be able to have a plenty of test cases instantly.
I already have an implementation of fn:transform(). I'll open its pull request after clearing housekeeping tasks, unless someone comes up with a better idea. Since it breaks compatibility here and there, introducing it as a replacement of the current version is clearly impossible. It will be an opt-in experimental feature.

Yes, I agree with you, fn:transform() should do the job, but it's an important amount of work to rewrite everything.
A question stays un-answered : is a private function in a package should be unit-tested ? On one hand, I'd be inclined to test everything ; on the other hand, a private function is not an API, and does not offer service to package end-users ; so it is a package implementation detail, and there is no imperial reason to unit-test it.
But, if everyone is Ok with non-testing private package function, this must be documented, as it will generate an error at runtime.

@jcronk @cmarchand , I submitted an experimental implementation: #892

I tried testing the code on the pull request, but it's entirely possible that I missed a step or I'm doing something wrong.  I initially tried it with several xslt packages, got an error, decided to take a step back and test it with something more minimal, and I got the same error.  Here's the minimal example I tried it with:
Package:
<xsl:package name="com.company.utils.array" package-version="0.1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:array-ext="http://www.company.com/utils/array"
    xmlns:array="http://www.w3.org/2005/xpath-functions/array" xmlns:fn="http://www.w3.org/2005/xpath-functions"
    xmlns:map="http://www.w3.org/2005/xpath-functions/map" exclude-result-prefixes="#all" expand-text="true" version="3.0">
    <xsl:expose component="function" names="array-ext:*" visibility="public"/>
    <xsl:function name="array-ext:interlace" as="array(*)*">
        <xsl:param name="first-item" as="item()*"/>
        <xsl:param name="second-item" as="item()*"/>
        <xsl:sequence
            select="
                for-each-pair($first-item, $second-item, function ($x, $y) {
                    [$x, $y]
                })"
        />
    </xsl:function>
    <xsl:function name="array-ext:shallow-flatten">
        <xsl:param name="ary" as="array(*)"/>
        <xsl:sequence select="(1 to array:size($ary)) ! $ary(.)"/>
    </xsl:function>
</xsl:package>

Stylesheet:
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:array-ext="http://www.company.com/utils/array" exclude-result-prefixes="#all" version="3.0">
    <xsl:output method="text" omit-xml-declaration="yes"/>
    <xsl:use-package name="com.company.utils.array" package-version="0.1.0"/>
    <xsl:variable name="nested-array" select="[1, 2, 3, 4, [5, 6], 7, [8, [9, 10]]]"/>
    <xsl:template name="output">
        <xsl:variable name="flattened-array" select="array-ext:shallow-flatten($nested-array)"/>
        <xsl:sequence select="serialize($flattened-array, map {'method': 'json'})"/>
    </xsl:template>
</xsl:stylesheet>

and config:
<configuration xmlns="http://saxon.sf.net/ns/configuration" edition="EE">
    <xsltPackages>
        <package name="com.company.utils.array" version="0.1.0" sourceLocation="array.xsl" />
    </xsltPackages>
</configuration>

In Oxygen XML, I'm getting:
     [java] Fatal error during transformation: java.lang.ClassCastException: net.sf.saxon.value.StringValue cannot be cast to net.sf.saxon.om.NodeInfo
When I run XSpec from the terminal, I get a somewhat more useful stack trace:
java.lang.RuntimeException: Internal error evaluating template rule  at line 17 in module file:/<snipped>/./xspec/Main-compiled.xsl
        at com.saxonica.ee.bytecode.ByteCodeCandidate.process(ByteCodeCandidate.java:153)
        at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:265)
        at net.sf.saxon.Controller.callTemplate(Controller.java:2547)
        at net.sf.saxon.s9api.Xslt30Transformer.callTemplate(Xslt30Transformer.java:750)
        at net.sf.saxon.Transform.processFile(Transform.java:1252)
        at net.sf.saxon.Transform.doTransform(Transform.java:782)
        at net.sf.saxon.Transform.main(Transform.java:81)
Caused by: java.lang.RuntimeException: Internal error evaluating template rule  at line 30 in module file:/<snipped>/./xspec/Main-compiled.xsl
        at com.saxonica.ee.bytecode.ByteCodeCandidate.process(ByteCodeCandidate.java:153)
        at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:265)
        at net.sf.saxon.expr.instruct.CallTemplate.process(CallTemplate.java:353)
        at net.sf.saxon.expr.instruct.CallTemplate.processLeavingTail(CallTemplate.java:411)
        at net.sf.saxon.expr.instruct.Block.processLeavingTail(Block.java:687)
        at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:151)
        at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:337)
        at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:284)
        at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:151)
        at net.sf.saxon.expr.instruct.ResultDocument.processInstruction(ResultDocument.java:480)
        at net.sf.saxon.Configuration.processResultDocument(Configuration.java:2273)
        at com.saxonica.config.EnterpriseConfiguration.processResultDocument(EnterpriseConfiguration.java:1938)
        at net.sf.saxon.expr.instruct.ResultDocument.process(ResultDocument.java:387)
        at net.sf.saxon.expr.instruct.ResultDocument.processLeavingTail(ResultDocument.java:373)
        at net.sf.saxon.expr.instruct.Block.processLeavingTail(Block.java:687)
        at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:151)
        at com.saxonica.ee.bytecode.ByteCodeCandidate.process(ByteCodeCandidate.java:141)
        ... 6 more
Caused by: java.lang.RuntimeException: Internal error evaluating template rule  at line 286 in module file:/<snipped>/xspec-testing/src/compiler/generate-tests-utils.xsl
        at com.saxonica.ee.bytecode.ByteCodeCandidate.process(ByteCodeCandidate.java:153)
        at net.sf.saxon.expr.instruct.NamedTemplate.expand(NamedTemplate.java:265)
        at net.sf.saxon.expr.instruct.CallTemplate.process(CallTemplate.java:353)
        at net.sf.saxon.expr.instruct.CallTemplate.processLeavingTail(CallTemplate.java:411)
        at net.sf.saxon.expr.instruct.Block.processLeavingTail(Block.java:687)
        at net.sf.saxon.expr.LetExpression.processLeavingTail(LetExpression.java:709)
        at net.sf.saxon.expr.instruct.Block.processLeavingTail(Block.java:687)
        at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:151)
        at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:337)
        at net.sf.saxon.expr.instruct.ElementCreator.processLeavingTail(ElementCreator.java:284)
        at net.sf.saxon.expr.instruct.Block.processLeavingTail(Block.java:687)
        at net.sf.saxon.expr.instruct.Instruction.process(Instruction.java:151)
        at com.saxonica.ee.bytecode.ByteCodeCandidate.process(ByteCodeCandidate.java:141)
        ... 22 more

The line number in Main-compiled.xsl that it's referencing is an xsl:message containing the scenario label, and the line number in generate-tests-utils.xsl is an xsl:param declaration under the test:report-sequence template.
I'm not sure if this is an issue with the code or an issue with something I'm doing.  In any case, I don't immediately see where the ClassCastException could be coming from, unless the line numbers it's giving are only approximations.
Up until the error hits, it looks like everything is working correctly - it seems to find the package and go through the initial steps just fine.  @AirQuick , if you see that I'm making a mistake somewhere, please let me know.

@jcronk Thanks for testing out the pr. Can you let us know these?

Saxon version
.xspec file
Corrected version of stylesheet. Looks like the current version has an error:C:\test>java -jar SaxonEE9-9-1-7J\saxon9ee.jar -it:output -xsl:test.xsl -config:config.xml
Error at char 10 in xsl:sequence/@select on line 9 column 85 of test.xsl:
  SERE0023: JSON output method cannot handle a sequence of more than one item

Did you run test/external_*.xspec successfully in the same environment?

Sure, sorry about that.
Corrected stylesheet:
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xs="http://www.w3.org/2001/XMLSchema"
    xmlns:array-ext="http://www.company.com/utils/array" exclude-result-prefixes="#all" version="3.0">
    <xsl:output method="text" omit-xml-declaration="yes"/>
    <xsl:use-package name="com.company.utils.array" package-version="0.1.0"/>
    <xsl:variable name="nested-array" select="[1, 2, 3, 4, [5, 6], 7, [8, [9, 10]]]"/>
    <xsl:template name="output">
        <xsl:variable name="flattened-array" select="array-ext:shallow-flatten($nested-array)"/>
        <xsl:sequence select="serialize($flattened-array, map {'method': 'adaptive'})"/>
    </xsl:template>
</xsl:stylesheet>

Should return something like
1
2
3
4
[5,6]
7
[8,[9,10]]

From Oxygen, the version of Saxon is <IP_ADDRESS> (EE), and the version I have XSpec pointing to on the command line is <IP_ADDRESS> (EE).
Here's the xspec file:
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec" stylesheet="Main.xsl" run-as="external">
    <x:variable name="x:saxon-config" select="'SaxonConf.xml'"/>
    <x:scenario label="Scenario for testing function output">
        <x:call template="output"/>
        <x:expect label="it does something" select="'foo'"/>
    </x:scenario>
</x:description>

I was expecting a failure here, obviously - I usually start with a test I know will fail so I can confirm what XSpec is getting for the result.  I thought maybe the string in the x:expect/@select attribute might have been what it was talking about in the error message, but if I put something in the content of x:expect it does the same thing.
I didn't think to run the external_* tests, but I just tried that from the command line and they were all successful; example:
Saxon script not found, invoking JVM directly instead.

Creating Test Stylesheet...

Running Tests...
Testing with SAXON EE <IP_ADDRESS>
Calling a public function in the package
should be possible

Formatting Report...
passed: 1 / pending: 0 / failed: 0 / total: 1
Report available at test/xspec/external_xslt-package_arith-result.html
Done.

<x:variable name="x:saxon-config" select="'SaxonConf.xml'"/>

This @select should be @href like this:
    <x:variable name="x:saxon-config" href="SaxonConf.xml"/>

or x:variable should contain a configuration element like this:
https://github.com/xspec/xspec/blob/f922358a21e40447a42e303f378cf655895c90ae/test/external_xslt-package_filter_use_static-param.xspec#L6-L16
I pushed some changes to the pr so that this kind of error is detected:
C:\xspec>bin\xspec.bat M:\test\test.xspec
...
Running Tests...
Testing with SAXON EE <IP_ADDRESS>
Scenario for testing function output
ERROR: Invalid $x:saxon-config

Thanks!  I corrected the variable to use href and now both of my basic examples work.  So far so good.

@AirQuick Here's a new one for you:
It looks like when I try to test higher-order functions, I get an error.  Here's an example of one:
Package file:
<xsl:package name="hof.reductions" xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
    xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:hof="http://www.company.com/hof"
    xmlns:math="http://www.w3.org/2005/xpath-functions/math" exclude-result-prefixes="xs math" version="3.0">
    <xsl:function name="hof:reductions" visibility="public">
        <xsl:param name="seq"/>
        <xsl:param name="init"/>
        <xsl:param name="func"/>
        <xsl:choose>
            <xsl:when test="empty($seq)">
                <xsl:sequence select="$seq"/>
            </xsl:when>
            <xsl:otherwise>
                <xsl:sequence select="$init, hof:reductions(tail($seq), $func($init, head($seq)), $func)"/>
            </xsl:otherwise>
        </xsl:choose>
    </xsl:function>
</xsl:package>

XSpec file:
<x:description xmlns:x="http://www.jenitennison.com/xslt/xspec" xmlns:hof="http://www.company.com/hof"
    stylesheet="reductions.xsl" run-as="external">
    <x:variable name="x:saxon-config" href="../../SaxonConf.xml"/>
    <x:scenario label="Scenario for testing function reductions">
        <x:call function="hof:reductions">
            <x:param name="seq" select="1, 2, 3, 4"/>
            <x:param name="init" select="0"/>
            <x:param name="func" select="function($res, $inp) { $res + $inp }"/>
        </x:call>
        <x:expect label="it returns a seq of each step in the reduction" select="0, 1, 3, 6"/>
    </x:scenario>
</x:description>

When I run this (same setup as before), I get this (the whole stack trace is really long, so this is just the first bit)
run-xslt-test:
     [echo] Running XSLT Tests...
     [java] Testing with SAXON EE <IP_ADDRESS>
     [java] Scenario for testing function reductions
     [java] java.lang.IllegalStateException: A function created under one Configuration cannot be called under a different Configuration
     [java] 	at com.saxonica.functions.hof.UserFunctionReference$BoundUserFunction.makeNewContext(UserFunctionReference.java:373)
     [java] 	at net.sf.saxon.functions.SystemFunction.dynamicCall(SystemFunction.java:451)
     [java] 	at net.sf.saxon.functions.ApplyFn.call(ApplyFn.java:159)
     [java] 	at net.sf.saxon.expr.FunctionCall.iterate(FunctionCall.java:547)
     [java] 	at net.sf.saxon.value.MemoClosure.makeSequence(Unknown Source)
     [java] 	at net.sf.saxon.value.MemoClosure.iterate(Unknown Source)
     [java] 	at net.sf.saxon.expr.VariableReference.iterate(VariableReference.java:540)
     [java] 	at net.sf.saxon.expr.instruct.BlockIterator.next(BlockIterator.java:49)
     [java] 	at net.sf.saxon.expr.instruct.BlockIterator.next(BlockIterator.java:51)
     [java] 	at net.sf.saxon.value.SequenceExtent.<init>(SequenceExtent.java:86)
     [java] 	at net.sf.saxon.value.SequenceExtent.makeSequenceExtent(SequenceExtent.java:111)
     [java] 	at net.sf.saxon.om.SequenceTool.toGroundedValue(SequenceTool.java:47)
     [java] 	at net.sf.saxon.s9api.Xslt30Transformer.callFunction(Xslt30Transformer.java:842)
     [java] 	at net.sf.saxon.functions.TransformFn.call(TransformFn.java:756)
     [java] 	at net.sf.saxon.expr.FunctionCall.iterate(FunctionCall.java:547)
     [java] 	at net.sf.saxon.expr.Expression.evaluateItem(Expression.java:843)
     [java] 	at net.sf.saxon.expr.parser.Evaluator$3.evaluate(Evaluator.java:73)
     [java] 	at net.sf.saxon.expr.parser.Evaluator$3.evaluate(Evaluator.java:70)
     [java] 	at net.sf.saxon.expr.SystemFunctionCall.evaluateArguments(SystemFunctionCall.java:448)
     [java] 	at net.sf.saxon.expr.FunctionCall.iterate(FunctionCall.java:545)
     [java] 	at net.sf.saxon.expr.LetExpression.iterate(LetExpression.java:501)
     [java] 	at net.sf.saxon.value.MemoClosure.makeSequence(Unknown Source)
     [java] 	at net.sf.saxon.value.MemoClosure.iterate(Unknown Source)

There's a comment here in the Saxon documentation that

From Saxon <IP_ADDRESS>, the vendor-options parameter allows an entry of the form "vendor-options": map {QName('http://saxon.sf.net/', 'configuration'): $config} where $config is the root node of a configuration file expressed as an XML tree. If this option is present, the target transformation is run in a different configuration from the calling application, so that settings in the configuration file do not affect the environment of the caller.

So this may not even be a bug - what I found after experimenting was that if I run XSpec from the terminal and I set SAXON_CUSTOM_OPTIONS with -config:MyConfig.xml and then remove the x:variable in the XSpec, it seems to run fine, but I'm not sure how I'd do that from inside Oxygen.
I've run into a couple of other things I wasn't expecting, but I'll need to gather more information to see whether it's worth reporting or if it's just behavior that's happening as a consequence of using packages.  Or if it's something about packages that I'm doing wrong, since this is pretty new to me.  So I'll update with more if I find anything else that seems noteworthy.  Thanks again for putting your branch out there!

I don't have an official answer from Saxonica, but it seems that if you use vendor-options (=$x:saxon-config), the invoked stylesheet can't call a function passed as a parameter, unfortunately.
So, if you want to call the parameterized function together with Saxon configuration specified, you probably have to use SAXON_CUSTOM_OPTIONS.

if I run XSpec from the terminal and I set SAXON_CUSTOM_OPTIONS with -config:MyConfig.xml and then remove the x:variable in the XSpec, it seems to run fine, but I'm not sure how I'd do that from inside Oxygen.

SAXON_CUSTOM_OPTIONS (command line) equivalence in Oxygen Run XSpec Test transformation scenario is saxon.custom.options as in the end of parameter list of this picture and explained in Wiki.

Just to update on this, @AirQuick, I've managed to get XSpec to run for a good portion of the codebase I'm converting to packages, and so far everything looks great, at least for my purposes.  I ran into an issue with visibility where if I used a function in the XSpec file that was defined in the SUT, it would say it couldn't be found.  But maybe I'm not supposed to do that, and in any case, I haven't run across it in a few days (but I'm still rebuilding the stylesheets and tests, since in order to get a fairly sparse example, I had to begin with a lot of what I'd previously had commented out).
I'm still working on this, but I want to reiterate my thanks for your answers to my dumb questions.  :)  There are a number of pieces I have yet to put together, so I'll update if I see any corner cases that look like they may be coming from the XSpec code changes.  The worst issue I've seen so far turned out to be an uncaught exception in Saxon itself, where if you use xsl:expose and you put an undeclared namespace prefix in the names attribute, it throws a null pointer exception and gives you no idea where it might be coming from.  Just mentioning this in case anyone else tests this branch and is as inattentive as I sometimes can be.

if I used a function in the XSpec file that was defined in the SUT, it would say it couldn't be found. But maybe I'm not supposed to do that

Right, as listed in the PR description:

In .xspec files (x:expect/@select etc.), you can't access variables/functions defined in the tested stylesheet.

If this is going to be a really annoying limitation, we might need to consider introducing something like x:include-helper which includes a helper stylesheet (or query) into the compiled stylesheet (or query).

an uncaught exception in Saxon itself, where if you use xsl:expose and you put an undeclared namespace prefix in the names attribute, it throws a null pointer exception and gives you no idea where it might be coming from.

I remember I encountered it myself while writing a test for this PR. I didn't have time to report it to Saxonica at that time, but yes, it actually was cryptic.

Right, as listed in the PR description:

In .xspec files (x:expect/@select etc.), you can't access variables/functions defined in the tested stylesheet.

Ah, okay.  I should have revisited the PR before posting about it.

If this is going to be a really annoying limitation, we might need to consider introducing something like x:include-helper which includes a helper stylesheet (or query) into the compiled stylesheet (or query).

I was actually going to ask about this separately, since it wasn't directly in the scope of the original issue: something like x:include-helper, or maybe a use-package that passes through to the compiled stylesheet, would be extremely helpful in some cases.  I've been using x:import to bring in a set of variables, some of which are anonymous functions, but being able to include a stylesheet (or use a package) would make things much nicer.


qtdesigner.path - Handle relative paths from workspace or venv

Qt Designer is now often used within the PyQt5-tools package (as you say into your readme) and it could be great to allow relative paths for the pyqt-integration.qtdesigner.path (and others):

from the ${workspaceFolder}
or from the python.pythonPath

Usually they are using PyQt5-tools or python distribution like Anaconda, if the command "designer" is available in your system, you can set it to pyqt-integration.qtdesigner.path and it should work.
If someone wants to put the PyQt5-tools package into ${workspaceFolder}, sounds interesting to add this support.

if use Anaconda, you can configure it, it works for me.
<anaconda path>/bin/Designer.app/Contents/MacOS/Designer


Error received when searching for records (May release)

I've installed the May release using the following procedure:
$ pipenv install invenio-cli
$ pipenv run invenio-cli --version  # returns invenio-cli, version 0.14.1
$ pipenv shell
$ invenio-cli init rdm
$ cd my-site
$ invenio-cli containerize --pre

Everything seems to have been installed correctly and completed successfully...
...
...
...

Creating files location...
Creating admin role...
Assigning superuser access to admin role...
Creating indices...
Collecting statics and assets...
Installing js dependencies...
Copying project statics and assets...
Building assets...
Instance running!
Visit https://<IP_ADDRESS>

$ invenio-cli demo --containers
$

...but when I looked at the running containers, I saw that Elasticsearch was unhealthy:
$ docker ps
CONTAINER ID        IMAGE                                                     COMMAND                  CREATED             STATUS                      PORTS                                                                                NAMES
a5b28f4215df        may-release-2-frontend                                    "nginx -g 'daemon of…"   18 minutes ago      Up 18 minutes               <IP_ADDRESS>:80->80/tcp, <IP_ADDRESS>:443->443/tcp, <IP_ADDRESS>:32793->80/tcp, <IP_ADDRESS>:32792->443/tcp      may-release-2_frontend_1
693b7b546d4a        may-release-2                                             "bash -c 'uwsgi /opt…"   18 minutes ago      Up 18 minutes               <IP_ADDRESS>:32791->5000/tcp                                                                      may-release-2_web-api_1
a653dfb28fca        may-release-2                                             "bash -c 'celery wor…"   18 minutes ago      Up 18 minutes                                                                                may-release-2_worker_1
34cd7e0ce491        may-release-2                                             "bash -c 'uwsgi /opt…"   18 minutes ago      Up 18 minutes               <IP_ADDRESS>:32790->5000/tcp                                                                      may-release-2_web-ui_1
e70d2f703327        rabbitmq:3-management                                     "docker-entrypoint.s…"   18 minutes ago      Up 18 minutes               4369/tcp, 5671/tcp, <IP_ADDRESS>:5672->5672/tcp, 15671/tcp, 25672/tcp, <IP_ADDRESS>:15672->15672/tcp   may-release-2_mq_1
0c3b9d446c9d        postgres:9.6                                              "docker-entrypoint.s…"   18 minutes ago      Up 18 minutes               <IP_ADDRESS>:5432->5432/tcp                                                                       may-release-2_db_1
37ef25a56179        redis                                                     "docker-entrypoint.s…"   18 minutes ago      Up 18 minutes               <IP_ADDRESS>:6379->6379/tcp                                                                       may-release-2_cache_1
a5fdaa00ca62        docker.elastic.co/elasticsearch/elasticsearch-oss:7.2.0   "/usr/local/bin/dock…"   18 minutes ago      Up 18 minutes (unhealthy)   <IP_ADDRESS>:9200->9200/tcp, <IP_ADDRESS>:9300->9300/tcp                                               may-release-2_es_1

I loaded https://<IP_ADDRESS>/ in the browser and was able to successfully search the demo data:

I ran several other searches of the demo data and didn't notice any errors or unexpected behavior. I then created my own record:

When I tried to search for this, I got an error:

There seemed to be a search indexing problem with Elasticsearch, because while I couldn't find the record with the text "Datalab" (shown above) or with my last name:

...I was able to find it with my first name:

As a side note, I ran through the install process a couple of times and while the search errors were similar in both cases, the specific search terms that worked vs. the ones that generated an error differed between the two tries.

Here are my Docker details...

and preferences...

and top screenshot:

This issue seems to have been resolved in the last release.


Detect anomalies in Cloud Audit Logs using BQML models

--- Adding a notebook to detect anomalies in Cloud Audit Logs using BQML models ---

- [✅ ] This notebook has been added to the [bigquery_ml](https://github.com/GoogleCloudPlatform/vertex-ai-samples/blob/main/notebooks/community/bigquery_ml) file under the `Community Notebooks` section

- [✅ ] Passes all the required formatting and linting checks.

@katiemn could you please help review this notebook? Thanks.


Margin on smaller screens

You have to work on margin for work section on smaller screens

For the larger screens, the yellow color like (button) is a little of(It is different in figma design).
For larger screens, the text and background have both background white, So you need to work on that.
For form textarea, you need to add required. (It is given in the requirements).
(just personal opinion, for the project image  you can improve the scaling of image like in viewpoint 780.  )


Workaround for SQLite decimal precision issues

SQLite's decimal type inaccuracies add up when summing over large numbers of values. Currently summation results can even be non-deterministic, being dependent on the order in which values arrived in the database.
At the root of this issue is SQLite's underlying use of floating point values in its decimal implementation.
As such we get rid of the decimal type and use integer values (satoshis) instead.
https://dev.azure.com/Stratisplatformuk/StratisBitcoinFullNode/_workitems/edit/4520.
This is the first change in what is currently expected to be a two-part fix.

Have checked that all of the decimal columns are indeed gone.

This change is a good one however, we must just remember that this again a breaking one and walles will need to be resynced.

Yes, Maciej is aware that this will only require deleting the wallet db files so that those can re-build themselves from the JSON files.


mingw64

xxxxx@DESKTOP-QV2HMLF MINGW64 /c/dpt-rp1-py-master
$ dptrp1 --client-id deviceid.dat --key privatekey.dat --addr <IP_ADDRESS> list-documents
Traceback (most recent call last):
File "C:/Users/gkhs1/AppData/Local/Programs/Python/Python37/Scripts/dptrp1", line 161, in
with open(args.client_id) as fh:
FileNotFoundError: [Errno 2] No such file or directory: 'deviceid.dat'
what problem? what should i do for solving ??
please answer . thanks

that is dptrp1 issues, please refer to their github


BUG: sorting with tuples and Multi-Index creation

Hello,
I'm using python 2.7.9 and pandas 0.15.2.
I'm trying to concatenate two DataFrames (to_concat is a list of 2 DataFrames) and I have the following error.
  File "/Users/alexisbenoist/Documents/CoTH/AFDS/app/core.py", line 67, in train_dataflow
    df = pd.concat(to_concat)
  File "/Users/alexisbenoist/.virtualenvs/afds/lib/python2.7/site-packages/pandas/tools/merge.py", line 725, in concat
    return op.get_result()
  File "/Users/alexisbenoist/.virtualenvs/afds/lib/python2.7/site-packages/pandas/tools/merge.py", line 896, in get_result
    mgrs_indexers, self.new_axes, concat_axis=self.axis, copy=self.copy)
  File "/Users/alexisbenoist/.virtualenvs/afds/lib/python2.7/site-packages/pandas/core/internals.py", line 4046, in concatenate_block_managers
    for placement, join_units in concat_plan]
  File "/Users/alexisbenoist/.virtualenvs/afds/lib/python2.7/site-packages/pandas/core/internals.py", line 4135, in concatenate_join_units
    empty_dtype, upcasted_na = get_empty_dtype_and_na(join_units)
  File "/Users/alexisbenoist/.virtualenvs/afds/lib/python2.7/site-packages/pandas/core/internals.py", line 4124, in get_empty_dtype_and_na
    raise AssertionError("invalid dtype determination in get_concat_dtype")
AssertionError: invalid dtype determination in get_concat_dtype

I put the pickle of the DataFrames in a gist using (I can upload it somewhere else if it's more convenient).
import pickle
with open(path, 'w+') as f:
    pickle.dump(to_concat, f)

I feel that I'm doing something wrong or it's a bug from pandas because if do:
pd.DataFrame(np.concatenate(to_concat), columns=to_concat[0].columns.tolist())

It gives a result which seems logical.
Any input is welcome.
Cheers,
Alexis.

Do you have a small script to generate two DataFrames that cause the error? That would be preferable to a pickle.
Also show exactly how you calling concat.

The script which generates the dataframes is a bit complicated but I can provide csv exports df1 df2.
I tried to reproduce the bug using, the csv and the dataframe but unfortunately it works.  I also tried to serialize the array using np.save, but it also works.
import pandas as pd
from ast import literal_eval as literal_eval_base

def literal_eval(s):
    try:
        return literal_eval_base(s)
    except Exception as e:
        return s

df_1 = pd.read_csv('df_1.csv')
df_2 = pd.read_csv('df_2.csv')

df_1.rename(columns=literal_eval, inplace=True)
df_2.rename(columns=literal_eval, inplace=True)
to_concat = [df_1, df_2]
pd.concat(to_concat)

I think that the bug is on the line 4039 of core/internals in the combine_concat_plans func.
Because, we can see below that the value in the iterator (JoinUnit(None, {}), JoinUnit(None, {})) is not the same as the one in the evaluated list (None of them is None).

I don't really know how i could send you the DataFrame in a readable format for you.

The get_mgr_concatenation_plan does not seems coherent as we can see on the picture below. When it's called by the code  (the length is 1) but when I call it from the debugger, the length is 6.

:+1:

This was a bit tricky to repro. However, the bottom line is their is a small edge case bug that you hit, though you should NOT ever use tuples in an index, unless it really is a multi-index, as these are by-definition the markers for multi-index guessing.
This is very odd in how things are sorted, not sure why exactly python does this (this is py2).
In [23]: sorted([u'class',(u'internal',u'__delete__')])
Out[23]: [(u'internal', u'__delete__'), u'class']

In [24]: sorted(['class',('internal','__delete__')])
Out[24]: ['class', ('internal', '__delete__')]

So this makes a determination if the guessed index is a multi-index or not (this is the bug part, I think we can be more strict, IOW we are only checking the first element if its a tuple; if it is we try to create a multi-index).
In [29]: Index([('A',1),'B'])
In [29]: Index([('A',1),'B'])
Out[29]:
MultiIndex(levels=[[u'A', u'B'], [1]],
           labels=[[0, 1], [0, -1]])

In [30]: Index(['B',('A',1)])
Out[30]: Index([u'B', (u'A', 1)], dtype='object')

Here's a repro example. Note that if these are strings (e.g. no u), then this would WORK!.
In [26]: df1 = DataFrame({ u'A' : 'foo', (u'B',1) : 'bar' },index=range(2))

In [27]: df2 = DataFrame({ u'B' : 'foo', (u'B',1) : 'bar' },index=range(2))

In [28]: pd.concat([df1,df2])
AssertionError: invalid dtype determination in get_concat_dtype

Thanks for the reactivity !

I'll close when I merge


Vsphere API Lookup Endpoints

With this PR we will start supporting vsphere cloning and we will upgrade our UI experience to allow for flags to be set in docker-machine. The field values for docker-machine are kind of cryptic and this is an issue to track adding in support for API endpoints to allow the UI to make requests into Vsphere and get answers to questions like "give me vms matching ubuntu-*" and populate selectors to make this process easier.
From a quick conversation with UI it sounds like we'll need an auth endpoint to validate cloud credentials and an endpoint to pull all data back from the vsphere server.
Fields we will want to have selectors for

data centers
hosts
content libraries
resource pools
data stores
folders?
hosts
networks

UI Issue: https://github.com/rancher/rancher/issues/23002

https://github.com/rancher/rancher/pull/23242

rancher/rancher:v2.3-head
UI: latest-2.3

Fields are populated by data from vCenter using endpoints:

Datacenter
Resource Pool
Data store
Folder
Host
Networks
Tags
Custom Attributes
Content libraries
Existing VMs
Templates


add ipv6 support

Changes mainframe Socket logic to support IPV6 hosting.
When connecting to a hostname, it will detect the AF_NET type and switch accordingly.

no


Idea for a standard way of importing Windows Trusted Root Certs for NodeJS

What: I am wondering if MS can come up with official way of doing this and make it part of some official documentation, like maybe here? https://docs.microsoft.com/en-us/windows/dev-environment/javascript/nodejs-on-windows#install-visual-studio-code

Why: Using Trusted Root Certificates in NodeJS program is cumbersome and there is no one official way of supporting it.
https://github.com/nodejs/node/issues/4175 and it is required for corp environments to make Secure Network Connection to outside.

Where: Somewhere on MS Official Docs for NodeJS

When: After discussion and with approvals?

Who: To MS Devs

How: Either use https://github.com/microsoft/vscode-windows-ca-certs or some other packages and promote it to have a standard way of importing Trusted Root Certs from Windows for all NodeJS and Electron communities

Background:
I was researching as to how to make our Electron app to fetch Trusted Root Certificates from Windows as it is not possible to know every trusted certificates from clients computer beforehand, and it seems to be that every Electron app uses different ways to solve this issue.
For vscode, I see that this is using
https://github.com/microsoft/vscode-windows-ca-certs from https://github.com/microsoft/vscode/blob/main/package.json (vscode-proxy-agent has dep to vscode-windows-ca-certs)
and it is a fork from
https://github.com/ukoloff/win-ca/
and I do see that This branch is 11 commits ahead, 269 commits behind ukoloff:master. from https://github.com/microsoft/vscode-windows-ca-certs
Since I couldn't make new issues on https://github.com/microsoft/vscode-windows-ca-certs, I was wondering if I can ask this to Microsoft as I know MS itself has other electron apps like MS Teams.
Maybe other Electron apps in MS is using other ways to import Trusted Root Certificates?

I like the idea, unfortunately it's currently beyond the scope of the VS Code project. /cc @deepak1556 in case we have someone familiar with Windows APIs and Node.js internally to pass this on.

I am not aware of contacts who can prioritize this one, best way forward is to work with the Node.js community https://github.com/nodejs/node/issues/3159#issuecomment-672747462.

Closing as out of scope. Thanks.


Rapid Test Profile onboarding not shown after app reset

Avoid duplicates

[X] Bug is not mentioned in the FAQ
[X] Bug is specific for Android only, for general issues / questions that apply to iOS and Android please raise them in the documentation repository
[X] Bug is not already reported in another issue

Technical details

Device name: Samsung Galaxy A5 (2017) SM-A520F
Android version: 8.0.0
App version: 2.7.1 (also in 2.9.0-RC0)

Describe the bug
After resetting the app no Rapid Test Profile onboarding screen is shown if a profile was created before the app was reset.
Steps to reproduce the issue

Uninstall CWA if already installed.
 (see screenshot below)
Tap "CONTINUE"
Enter "A" in field "First Name", tap "SAVE"
Tap "X", tap "X"
Tap three-dot icon > Settings > Reset App and confirm
Repeat onboarding steps 3 - 7
Repeat steps 8 and 9 to Create Rapid Test Profile
Notice this time that no onboarding screen appears. Instead a blank Rapid Test Profile is immediately presented.

Expected behavior
After resetting the app the Create Rapid Test Profile onboarding screen should always be shown when accessing this function for the first time after the app is reset.
Possible Fix
Ensure that resetting the app causes the status of the Create Rapid Test Profile onboarding to be cleared so that it is shown.

FYI: This issue is addressed by #3970

@marcauberer
Unfortunately Google has not yet allowed me to access the final 2.9 build from Google Play Store yet, due to the staged rollout. I'm happy in general to verify with the Google Play Store build and then close the issue when I've done so.
In this case you got there before me.

Successfully verified with CWA 2.9.0 on device name: Samsung Galaxy A5 (2017) SM-A520F, Android version: 8.0.0, therefore confirming closure of this issue.
Steps to verify

Uninstall CWA if already installed.

Tap "CONTINUE"
Enter "A" in field "First Name", tap "SAVE"
Tap "X", tap "X"
Tap three-dot icon > Settings > Reset App and confirm
Repeat onboarding steps 3 - 7
Repeat steps 8 and 9 to Create Rapid Test Profile
Note that onboarding screen correctly appears, as in step 10.

@MikeMcC399 @PhilippNowak96 Thanks for contributing.


Nepal's Data

Nepal’s Data Added
Added Nepal at db.js

Looks great! Thanks @nikeshbalami!

Yay Yay thanks @todrobbins


feat!: release version 5.0.0

https://github.com/asyncapi/spec-json-schemas/pull/386 did work
I finally learned why. There release config was in a wrong location so semantic-release applied defaults, which is for conventional commits it used angular way that do not respect feat! or fix! but requires explicit BREAKING CHANGE:
now this release config is globally fixed everywhere and in the root of the repo

/rtm


Is this still alive?

Great project all in all! I love getting rid of jquery in angular projects which is why I came looking for this. However there are some pretty killer bugs (picker dying with existing model values for one). Are these things going to be fixed?

https://github.com/alongubkin/angular-datepicker/commit/4737234dbfb3da67e85e53bf1bff378469d83db6


ISPN-1768 Avoid creating unnecessary Integer instances in AbstractWheelC...

...onsistentHash
https://issues.jboss.org/browse/ISPN-1768

Integers between -128 and 127 are actually allocated only once by the JVM and cached. What are the savings here ?

Well, it's not the integers - the structure created was entirely unnecessary. Have a look at the patch. :)
Sent via the InterWebs
On 22 Jan 2012, at 13:42<EMAIL_ADDRESS>wrote:

Integers between -128 and 127 are actually allocated only once by the JVM and cached. What are the savings here ?

Reply to this email directly or view it on GitHub:
https://github.com/infinispan/infinispan/pull/873#issuecomment-3601762

@maniksurtani, Sanne did the exact same thing 8 hours ago :)
@tristantarrant, since the hash space is now from 0 to Integer.MAX_VALUE, the chances of a hash id falling between 0 and 127 are quite small.

@maniksurtani yes, no need to steal my issue since I'd already sent a pull request! :)

Well, glad to see the code is identical!
https://github.com/infinispan/infinispan/pull/872
I suppose it means we agree on it.

ROFL!  Hilarious! :)

merged the first one ;)


Add SerenityOS crossbuild

Cross compile mruby for SerenityOS (https://github.com/SerenityOS/serenity)
This goes along with the port I wrote for serenityOS

Thank you! I will merge the PR.


Title support

Is it possible to add a title to the input?

Actually, I tried it out and see that it is. It'd be nice to add to the documentation that you can pass through field values like this. It wasn't clear to me from the docs, so I almost didn't try Cleave


How to import jobsub to workflow v2？

Please support importing jobsub to workflow v2.

I have sovled it temporarily. Make josub to an action,  similar to sub-workflow,  internal implement using sub-workflow

Yes, this is a good alternative for now!
On Sun, May 15, 2016 at 7:28 PM linflytang<EMAIL_ADDRESS>wrote:

I have sovled it temporarily. Make josub to an action, similar to
sub-workflow, internal implement using sub-workflow
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub
https://github.com/cloudera/hue/issues/365#issuecomment-219336213


use begin end have exception

Using jsqlparser version
0.9.5
batch update from mybatis
begin
update table1 set a = 'xx' where b = 'condition1';
update table1 set a = 'xx' where b = 'condition2';
end;
Have Exception

begin and end are not implemented. This could be done within the statements parser.


Solutions for P01-P06

Hi @eneko
Can you give me a hint on what's the issue with travis? It still fails even after I've pushed the sources.

Going to restart the build, not sure why it failed. Seems like a permissions issue, I might need to check how Danger works with forked repositories.

Hi @alexmarchis1990,
Seems like the build is failing now because the List initializers no longer return optional. Please see if you can fix that. Regarding the warnings, please ignore those, I was doing some tests with SwiftLint.

Hi @eneko
I've made the required corrections and also added a solution for P07. Let me know if any problems, thanks!

Awesome, good job!


Data download not working

It seems like the data download is not working.
For example:
wget http://s3.amazonaws.com/proteindata/data_pytorch/pfam.tar.gz
terminates with a HTTP 403：Forbidden
and the other data is the same.

Hi @Gary-Deeplearning:
Thank you for the feedback. I have updated the download address of the datasets. If there is any other problems, please feel free to contact me.
Best,
Yijia Xiao


Test/add jest auto test

:tada: This PR is included in version 1.16.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


[SPARK-12564] [SQL] Improve missing column AnalysisException

org.apache.spark.sql.AnalysisException: cannot resolve 'value' given input columns text;

lets put a : after columns and put the columns in [] so that they match the toString of DataFrame.

Test build #48441 has started for PR 10518 at commit bc57b3a.

LGTM.

Test build #2261 has started for PR 10518 at commit bc57b3a.

Test build #2261 has finished for PR 10518 at commit bc57b3a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test build #48441 has finished for PR 10518 at commit bc57b3a.

This patch fails from timeout after a configured wait of `250m`.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/48441/
Test FAILed.

I've merged this. Thanks!

Thanks!


Calculator

Add frontend page for the calculator (separate page).  Conversion between two currencies is the primary feature.

[x] Add drop down menus for start and end currencies
[x] Add api calls for calculatorData
[x] Display rate conversion automatically

Close as of 93e32a49d78acfd17036e48a93620f2ba4aee179 and 6db9b93b9ed735ba67f8d3eefc19b74796c4c4b9


Documentation about bootstrap_endpoint of confluent_cluster_link is invalid

According to a page of confluent_cluster_link, bootstrap_endpoint argument of source_kafka_cluster block accepts both format like SASL_SSL://XXX:9092 and XXX:9092.
But in fact, it accepts only a format starting with SASL_SSL://.
My codes is below:
resource "confluent_cluster_link" "destination-outbound" {
  link_name = "link_name"

  source_kafka_cluster {
    id                 = "my_cluster_id"
    bootstrap_endpoint = "XXX.XXX.XXX.XXX:9092" # raw IP address
  }

  destination_kafka_cluster {
    id            = confluent_kafka_cluster.destination-cluster.id
    rest_endpoint = confluent_kafka_cluster.destination-cluster.rest_endpoint
    credentials {
      key    = confluent_api_key.app-manager-api-key-destination.id
      secret = confluent_api_key.app-manager-api-key-destination.secret
    }
  }
}

And the error message is below:
% terraform plan
╷
│ Error: invalid value for source_kafka_cluster.0.bootstrap_endpoint (the bootstrap endpoint must start with 'SASL_SSL://')
│
│   with confluent_cluster_link.destination-outbound,
│   on link_and_mirror.tf line 6, in resource "confluent_cluster_link" "destination-outbound":
│    6:     bootstrap_endpoint = "XXX.XXX.XXX.XXX:9092"
│
╵
%

@yusaito01 could you confirm you're using TF Provider for Confluent with version 1.9.0 or later? The issue you're referencing seems like a duplicate of #118 that we should have fixed in 1.9.0.

It doesn't look like there're any validations for SASL_SSL:// prefix left in resource_cluster_link.go.

@linouk23 Thank you for your comment.

could you confirm you're using TF Provider for Confluent with version 1.9.0 or later?
sorry I used TF Provider version 1.5.1. After update to 1.10.0, this error was resolved.


Create CODEOWNERS

The team name may need to be prefixed with the org: @puppetlabs/edu-tools-maintainers. I did that when I added CODEOWNERS to the ILT content repositories a while ago, but maybe that needs to be verified.


⚠ Processors has degraded performance

In b70cbf9, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 57 ms

Resolved: Processors performance has improved in 809e565.


Add button to "ztp status" page to uninstall AOS run files

invoke aos-installer-foo uninstall command from webui.

@bobbywatson3 my request for this as well is to send it to an 'uninstall' page with the device IP pre-filled: this way I can run the AOS uninstaller on a device that may not be currently registered to AOS.
I can write the UI part of this request if you've already started the uninstaller.

closing this as it is AOS specific.


alternative to inetutils

using preferrednetworks  in inetutils we were able to register network ip address on which the service is running in spring apps.
is there any alternative to achieve this for node apps.
https://github.com/spring-cloud/spring-cloud-commons/blob/master/docs/src/main/asciidoc/spring-cloud-commons.adoc#ignore-network-interfaces

I tried to pass dataCenterInfo.metadata with my local-ipv4 but this:

https://github.com/jquatier/eureka-js-client/blob/master/src/EurekaClient.js#L530

Is actually overwriting all my configs. Which doesn't make sense.
The arguments should be passed to the merge function in the opposite order.


video_player image distortion problem after last flutter update 0.11.3

After the last update of flutter I have begun to have problems in some android devices with the video plugin.
It would seem like a codec problem.
To rule out a problem in my project, create a new one with the base example of the video plugin.
I attach a screenshot.

The device I use testing is a one plus one with android 8, I could also try an apk on a Samsung Galaxy S8 and an iphone s6 without problems.
Flutter doctor:
[flutter] flutter doctor
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel beta, v0.11.3, on Mac OS X 10.13.6 17G3025, locale en-AR)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
[✓] Android Studio (version 3.2)
[✓] VS Code (version 1.29.0)
[✓] Connected device (2 available)

• No issues found!
exit code 0

Debug Console:
Launching lib/main.dart on A0001 in debug mode...
Built build/app/outputs/apk/debug/app-debug.apk.
I/ExoPlayerImpl(14246): Init 676cb13 [ExoPlayerLib/2.8.0] [bacon, A0001, OnePlus, 27]
I/ExoPlayerImpl(14246): Init 8c4cf49 [ExoPlayerLib/2.8.0] [bacon, A0001, OnePlus, 27]
D/NetworkSecurityConfig(14246): No Network Security Config specified, using platform default
W/VideoCapabilities(14246): Unrecognized profile/level 0/3 for video/mpeg2
W/VideoCapabilities(14246): Unrecognized profile/level 0/3 for video/mpeg2
W/VideoCapabilities(14246): Unrecognized profile<PHONE_NUMBER> for video/avc
I/VideoCapabilities(14246): Unsupported profile 4 for video/mp4v-es
D/SurfaceUtils(14246): connecting to surface 0x8edda008, reason connectToSurface
I/MediaCodec(14246): [OMX.qcom.video.decoder.avc] setting surface generation to 14587905
D/SurfaceUtils(14246): disconnecting from surface 0x8edda008, reason connectToSurface(reconnect)
D/SurfaceUtils(14246): connecting to surface 0x8edda008, reason connectToSurface(reconnect)
D/SurfaceUtils(14246): set up nativeWindow 0x8edda008 for 560x320, color 0x7fa30c04, rotation 0, usage 0x42002900
D/vndksupport(14246): Loading<EMAIL_ADDRESS>from current namespace instead of sphal namespace.
D/vndksupport(14246): Loading /vendor/lib/hw/gralloc.msm8974.so from current namespace instead of sphal namespace.
W/GrallocMapperPassthrough(14246): buffer descriptor with invalid usage bits 0x2002000
I/chatty  (14246): uid=10092(com.example.videotest) CodecLooper identical 4 lines
W/GrallocMapperPassthrough(14246): buffer descriptor with invalid usage bits 0x2002000
I/ACodec  (14246): codec does not support config priority (err -2147483648)
D/SurfaceUtils(14246): set up nativeWindow 0x8edda008 for 560x320, color 0x7fa30c04, rotation 0, usage 0x42002900
D/AudioTrack(14246): Client defaulted notificationFrames to 2362 for frameCount 7088
D/vndksupport(14246): Loading /vendor/lib/hw/gralloc.msm8974.so from current namespace instead of sphal namespace.
D/vndksupport(14246): Loading /vendor/lib/hw/gralloc.msm8974.so from current namespace instead of sphal namespace.
I/zygote  (14246): Do partial code cache collection, code=29KB, data=29KB
I/zygote  (14246): After code cache collection, code=29KB, data=29KB
I/zygote  (14246): Increasing code cache capacity to 128KB
D/vndksupport(14246): Loading /vendor/lib/hw/gralloc.msm8974.so from current namespace instead of sphal namespace.
I/chatty  (14246): uid=10092(com.example.videotest) Thread-2 identical 2 lines
D/vndksupport(14246): Loading /vendor/lib/hw/gralloc.msm8974.so from current namespace instead of sphal namespace.
W/OkHttpClient(14246): A connection to http://www.sample-videos.com/ was leaked. Did you forget to close a response body?
I/zygote  (14246): Do partial code cache collection, code=61KB, data=53KB
I/zygote  (14246): After code cache collection, code=61KB, data=53KB
I/zygote  (14246): Increasing code cache capacity to 256KB
W/OkHttpClient(14246): A connection to https://www.sample-videos.com/ was leaked. Did you forget to close a response body?
W/OkHttpClient(14246): A connection to http://www.sample-videos.com/ was leaked. Did you forget to close a response body?
W/OkHttpClient(14246): A connection to http://www.sample-videos.com/ was leaked. Did you forget to close a response body?
I/zygote  (14246): Do full code cache collection, code=123KB, data=88KB
I/zygote  (14246): After code cache collection, code=114KB, data=70KB
E/ExoPlayerImplInternal(14246): Source error.
E/ExoPlayerImplInternal(14246): com.google.android.exoplayer2.upstream.HttpDataSource$InvalidResponseCodeException: Response code: 404
E/ExoPlayerImplInternal(14246): 	at com.google.android.exoplayer2.upstream.DefaultHttpDataSource.open(DefaultHttpDataSource.java:211)
E/ExoPlayerImplInternal(14246): 	at com.google.android.exoplayer2.source.ExtractorMediaPeriod$ExtractingLoadable.load(ExtractorMediaPeriod.java:848)
E/ExoPlayerImplInternal(14246): 	at com.google.android.exoplayer2.upstream.Loader$LoadTask.run(Loader.java:317)
E/ExoPlayerImplInternal(14246): 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
E/ExoPlayerImplInternal(14246): 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
E/ExoPlayerImplInternal(14246): 	at java.lang.Thread.run(Thread.java:764)
I/flutter (14246): Video player had error com.google.android.exoplayer2.ExoPlaybackException
W/OkHttpClient(14246): A connection to https://www.sample-videos.com/ was leaked. Did you forget to close a response body?
W/OkHttpClient(14246): A connection to http://www.sample-videos.com/ was leaked. Did you forget to close a response body?
W/OkHttpClient(14246): A connection to https://www.sample-videos.com/ was leaked. Did you forget to close a response body?
I/zygote  (14246): Do partial code cache collection, code=125KB, data=82KB
I/zygote  (14246): After code cache collection, code=125KB, data=82KB
I/zygote  (14246): Increasing code cache capacity to 512KB
I/zygote  (14246): Background concurrent copying GC freed 11255(4MB) AllocSpace objects, 7(476KB) LOS objects, 60% free, 3MB/9MB, paused 5.033ms total 71.098ms

I reproduced this on a Samsung Galaxy S5.
This was introduced by https://github.com/google/skia/commit/dc82994e3a76141cc037c92a644b7a6c54c87ad7
That commit has some special cases for specific platforms and GPUs.  Possibly more devices need to be treated specially?
@bsalomon

I have the same problem，my device is one plus x Android 6.0，flutter 0.11.3

I can confirm I get the exact same issue on an LG G2 (D802) running kik kat 4.4.2. However, I don't see this problem on an Android Galaxy S6.
Some other issues I believe are actually the same: https://github.com/flutter/flutter/issues/24289#issuecomment-438998391 and  https://github.com/flutter/flutter/issues/23930#issuecomment-439273322

Fixed by https://skia.googlesource.com/skia/+/a6dddeaac902114677b21cb39f423db0f5557b10 (as soon as that rolls into Flutter)

Thanks! I will wait for the update

@lgArlequin this should be fixed in the latest release (v0.11.10, available on the dev and beta channels).  Can you please run flutter upgrade and then verify to see if it's fixed for you?

@tvolkert We are still facing image distortion problem with v0.11.10 on Smartisan YQ601.
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel dev, v0.11.10, on Mac OS X 10.14.1 18B75, locale en-CN)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
[✓] Android Studio (version 3.2)
[✓] IntelliJ IDEA Ultimate Edition (version 2018.3)
[✓] VS Code (version 1.29.1)
[✓] Connected device (2 available)

• No issues found!

  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41
    creation_time   : 2018-11-19T09:37:19.000000Z
  Duration: 00:00:11.05, start: 0.000000, bitrate: 1789 kb/s
    Stream #0:0(eng): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv, bt709), 750x1334, 1739 kb/s, 30 fps, 30 tbr, 30k tbn, 60 tbc (default)
    Metadata:
      creation_time   : 2018-11-19T09:37:19.000000Z
      handler_name    : ?Mainconcept Video Media Handler
      encoder         : AVC Coding
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 45 kb/s (default)
    Metadata:
      creation_time   : 2018-11-19T09:37:19.000000Z
      handler_name    : #Mainconcept MP4 Sound Media Handler

@lgArlequin this should be fixed in the latest release (v0.11.10, available on the dev and beta channels). Can you please run flutter upgrade and then verify to see if it's fixed for you?

@tvolkert, no, unfortunately the problem continues.
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel beta, v0.11.10, on Mac OS X 10.13.6 17G3025, locale en-AR)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
[✓] Android Studio (version 3.2)
[✓] VS Code (version 1.29.1)
[✓] Connected device (2 available)

Ok, second take... can you please try v0.11.11 (on the latest dev and beta channels - available via flutter upgrade)?  Thanks!

same for Nexus 5, Android 5. still affected.
[✓] Flutter (Channel beta, v0.11.11, on Linux, locale en_US.UTF-8)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)
[✓] Android Studio (version 3.2)
[✓] Connected device (1 available)

I have upgrade to dev 0.10.12，and I tried with  Le X620（Android 6.0），MHA-AL00（Android 8.0）,
in this two phone model. But in Che1-CL20（Android4.4.4），the problem still exists。

@hyz1992 thanks for your patience!  We continue to look into this, and I'm about to release 0.11.13, with another shot at fixing this.

Great BUT.  I have seen this bug too, but I would assume the #24565 bug is FAR more important than this one as using newer Android devices, this issue is not a problem..  The fact #24565 means video player in next to useless even if this is fixed is the key issue here.   I hope your addressing #24565 too.

@jamiegau thanks for the heads up on #24565.  We know the exact commit that introduced this bug, which is why we're trying to fix this while it's still fresh in our minds.

Great, once I see a commit on this I will do some testing from my side.  Reliability is key on these features.  I'll set up a emulator and android device and run it 24/7 in my office and keep an eye on it while I work.  flutter having this feature with a solid and reliable capability is a key feature for many.

Ok, v0.11.13 is live on the dev and beta channels, with the latest attempt at fixing this.

@tvolkert Thanks for the fix, I can confirm that the green badge issue has been fixed in v0.11.13 on Smartisan YQ601, but the image rendering quality does not seem good. Is there anything I can do to improve the quality?
Doctor summary (to see all details, run flutter doctor -v):
[✓] Flutter (Channel dev, v0.11.13, on Mac OS X 10.14.1 18B75, locale en-CN)
[✓] Android toolchain - develop for Android devices (Android SDK 28.0.3)
[✓] iOS toolchain - develop for iOS devices (Xcode 10.1)
[✓] Android Studio (version 3.2)
[✓] IntelliJ IDEA Ultimate Edition (version 2018.3)
[✓] VS Code (version 1.29.1)
[✓] Connected device (3 available)

• No issues found!

Yes, it has been fixed in  Che1-CL20（Android4.4.4）,Thank you very much!

Thanks @microdog.  Would you mind filing a separate issue for that so that this issue doesn't get overloaded?
I'll close this one out - thanks everyone!

It is working now. Thanks @tvolkert

Thanks @tvolkert , I also confirm the fix!

I am getting the issue still in the latest android devices(Android 10).


Sheet: Using open and onOpenChange props on Sheet component causes the application to freeze and break

<Button variant='outline' onClick={() => setOpenSheet(true)}>             Filters           </Button>           {openSheet && (             <FilesFilterSheet               onApply={(data: any) => filteredData(data)}               openSheet={openSheet}             />           )}

how do u solved this?

OH, I solved putting this on my global css
body {
pointer-events: auto !important;
}
The shadcn Sheet is changing to none...


TPU profiling extension

TensorBoard startup is extended with TPU profiling option (in case of TPU is present in the Colab environment)
See: https://cloud.google.com/tpu/docs/cloud-tpu-tools

I think this TPU error has nothing to do with my modifications. In my experience it can be due to compatibility issues between the installed Tensorflow and Keras versions. A solution is to import the Keras pieces from tensorflow.python.keras . I have modified it accordingly, could you give it a try?


Resolve the problem of the pygame error: File is not a Windows BMP file

Solution found here -> http://www.akeric.com/blog/?tag=kivy

Hi - I am working with kivy in oracle linux 64-bit and I was receiving the error:
pygame.error: File is not a Windows BMP file
Here is the full error text I receive BEFORE making the change listed in this issue:
'''
[ [1;32mINFO [0m   ] [Window      ] Provider: pygame(['window_egl_rpi'] ignored)
[ [1;31mERROR [0m  ] [WinPygame   ] unable to set icon
Traceback (most recent call last):
File "/usr/local/lib/python3.4/site-packages/kivy/core/window/window_pygame.py", line 215, in set_icon
self._set_icon_standard(filename)
File "/usr/local/lib/python3.4/site-packages/kivy/core/window/window_pygame.py", line 227, in _set_icon_standard
im = pygame.image.load(filename)
pygame.error: File is not a Windows BMP file
[ [1;33mWARNING [0m] [WinPygame   ] Video: failed (multisamples=2)
[ [1;33mWARNING [0m] [WinPygame   ] trying without antialiasing
[ [1;32mINFO [0m   ] [GL          ] OpenGL version <b'2.1 Mesa 10.4.3'>
[ [1;32mINFO [0m   ] [GL          ] OpenGL vendor <b'Mesa Project'>
[ [1;32mINFO [0m   ] [GL          ] OpenGL renderer <b'Software Rasterizer'>
[ [1;32mINFO [0m   ] [GL          ] OpenGL parsed version: 2, 1
[ [1;32mINFO [0m   ] [GL          ] Shading version <b'1.20'>
[ [1;32mINFO [0m   ] [GL          ] Texture max size <16384>
[ [1;32mINFO [0m   ] [GL          ] Texture max units <32>
[ [1;32mINFO [0m   ] [Window      ] virtual keyboard not allowed, single mode, not docked
'''

After I make the change listed in this issue here is the full error text I am receiving:
[ [1;32mINFO [0m   ] [Logger      ] Record log in /root/.kivy/logs/kivy_16-01-10_6.txt
[ [1;32mINFO [0m   ] [Kivy        ] v1.9.1
[ [1;32mINFO [0m   ] [Python      ] v3.4.3 (default, Jan  9 2016, 16:11:53)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-16)]
[ [1;32mINFO [0m   ] [Factory     ] 179 symbols loaded
[ [1;32mINFO [0m   ] [Image       ] Providers: img_tex, img_dds, img_gif, img_pygame, img_pil (img_ffpyplayer ignored)
Warning (from warnings module):
File "/usr/local/lib/python3.4/site-packages/kivy/core/text/text_pygame.py", line 24
pygame.font.init()
RuntimeWarning: use font: No module named 'pygame.font'
(ImportError: No module named 'pygame.font')
[ [1;32mINFO [0m   ] [Text        ] Provider: pil(['text_pygame'] ignored)
...then at the end of the program:
[ [1;31mCRITICAL [0m] [Window      ] Unable to find any valuable Window provider at all!
egl_rpi - ImportError: cannot import name 'bcm'
File "/usr/local/lib/python3.4/site-packages/kivy/core/init.py", line 59, in core_select_lib
fromlist=[modulename], level=0)
File "/usr/local/lib/python3.4/site-packages/kivy/core/window/window_egl_rpi.py", line 12, in
from kivy.lib.vidcore_lite import bcm, egl
pygame - TabError: inconsistent use of tabs and spaces in indentation (window_pygame.py, line 203)
File "/usr/local/lib/python3.4/site-packages/kivy/core/init.py", line 59, in core_select_lib
fromlist=[modulename], level=0)
x11 - ImportError: No module named 'kivy.core.window.window_x11'
File "/usr/local/lib/python3.4/site-packages/kivy/core/init.py", line 59, in core_select_lib
fromlist=[modulename], level=0)
[ [1;31mCRITICAL [0m] [App         ] Unable to get a Window, abort.

Please let me know what additional information needed to help resolve this. Thank you.

I'm working in python 3.4.3 btw.

I don't know how to fix it for your version of python because i'm using 2.7. But i recommend to use SDL2 instead of pygame, unfortunately i can't use it on centos 7 so i still use pygame.

@krusaderjake you've mixed tabs and spaces during the chage and py3 fails in that case.
@Farkal have you tried installing sdl2? It seems there are packages available: https://rpmfind.net/linux/rpm2html/search.php?query=SDL2-devel&submit=Search+...&system=&arch=.
Or you could compile the source.
As for the pr, it strips out Windows ico loading, what is the reason?

Thanks for your answer :smile:
@dessant yes i have tried, I have also build SDL2 from source but i can't find the source of SDL2-devel and I think that the SDL2-devel on the centos repos is too old. Do you think it's safe to try to install Fedora packages on CentOS 7 ?
For the pr the goal was to remove the error message with windows ico as the author say here -> http://pythonwiki.tiddlyspot.com/#[[Kivy %3A Debugging via Wing]]

The error does not happen at win ico handling, but in _set_icon_standard, you could handle the exception there.
When i've compiled on debian, using this source seemed to be enough https://www.libsdl.org/release/SDL2-2.0.4.zip, and the rest of the sdl2 deps must be installed too, sdl2_ttf, sdl2_image, sdl2_mixer.
I have no idea about package safety across those distros, I have not used them yet. :-P

Ok but is there a way to remove this error ?
I try to compile fedora packages on centOS and i successfully compile SDL 2.0.4 but it's impossible to compile SDL-devel 2.0.4 (not the same window manager as fedora) so i will wait for the official release.

The pygame providers have been deprecated, but if someone provides the right fix for this issue, we can merge it.


Not working in mobile version

Hi, in order to evaluate this, much more info will be needed.
Ideally a demo that shows it not working and what environment to use to reproduce


Adds feature to let users opt-in to have notifications forwarded via email.

Adds feature to let users opt-in to have notifications forwarded via email.

@Polsaker May I ask for your opinion on this one? Since this will require more works (i.e. tests and some polish) I'd like to know beforehand if it's shaping into something you could find acceptable.

Sorry for the delay, this looks good to merge now

Sorry for the delay, this looks good to merge now

Thanks, let me know if there's some post-merge work to do on that in case you'd get some feedbacks from users.

There was a missing migration that I already added. Will let you know if anything else happens

@why-not-try-calmer in app/config.py can you maybe make allow_email_forwarded_notifications a part of configurable_defaults instead of defaults? This way we don't have to mess with it inside config.yaml (we also now miss an updated example_config.yaml, by the way).
Thanks!

Sounds like a reasonable request.

For now I'm testing with the config.yaml option set to True and getting:
Traceback (most recent call last):
  File "/home/app/.pyenv/versions/app/lib/python3.9/site-packages/flask/app.py", line 2531, in wsgi_app
    response = self.handle_exception(e)
  File "/home/app/.pyenv/versions/app/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/app/.pyenv/versions/app/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/app/.pyenv/versions/app/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/app/.pyenv/versions/app/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
  File "/home/app/.pyenv/versions/app/lib/python3.9/site-packages/flask_login/utils.py", line 290, in decorated_view
    return current_app.ensure_sync(func)(*args, **kwargs)
  File "/home/app/ceknito/app/views/user.py", line 331, in edit_user
    return engine.get_template("user/settings/preferences.html").render(
  File "/home/app/ceknito/app/misc.py", line 136, in render
    result = super(TemplateWithLogging, self).render(ctx)
  File "/home/app/.pyenv/versions/app/lib/python3.9/site-packages/wheezy/template/engine.py", line 28, in render
    return self.render_template(ctx, {}, {})
  File "user/settings/preferences.html", line 64, in render
  File "/home/app/.pyenv/versions/app/lib/python3.9/site-packages/wheezy/template/engine.py", line 70, in render
    return self.renders[name](ctx, local_defs, super_defs)
  File "shared/layout.html", line 178, in render
  File "shared/layout.html", line 169, in content
  File "user/settings/preferences.html", line 39, in main
AttributeError: 'EditUserForm' object has no attribute 'email_forwarded_notifications'

After I've managed to fix my pending PR, we can look more into it.

Seeing as your pending PR is still not fixed, can you maybe advise on the above error so that I can at least use your previous PR (and the other PRs introduced since then by Polsaker)?


Added reading of beads and bonded terms in XMLTopologyReader

Modifications:

switched to XML parser based on Property tree
new section molecule under <molecules> that stores <bead>

    <molecules>
        <molecule name="BUT" nmols="4000" nbeads="4">
            <bead name="C1" type="C" mass="15.035" q="0.0" />
            <bead name="C2" type="C" mass="14.028" q="0.0" />
            <bead name="C3" type="C" mass="14.028" q="0.0" />
            <bead name="C4" type="C" mass="15.035" q="0.0" />
        </molecule>
    </molecules>

new section <bonded> to store bonded terms

<bonded>
    <bond>
        <name>bond1</name>
        <beads>
            BUT:C1 BUT:C2
        </beads>
    </bond>
</bonded>

Example of topology for butane (UA) molecules: topol.xml

@ruehle @symashayak : This is a big change.
My understand is that you want to have a more flexible molecule definition in topol.xml, right? And currently things can only be renamed from a base topology, which not xml, right?.Additionally bonds have to be defined in the mapping file, which is a bit awkward.
This needs to be documented in the manual and a new file topol.xml in  share/xml.
As you comment is changed, did you find the bug in csg_stat not using the exclusions correctly?

I still work on that, but I want to share it to discuss the approach before I write more code ;-) The exclusions seem to work correctly, I missed something.
The idea is to get rid of using .gro, .pdb as a base and have at least partially description of the system in single file. Recently we had to play with lammps and we had issue with naming of atoms in the molecules, so then some mapping could be done. Lammps does not have anything like that in dump file and xml can only do rename of molecules. In the end we use .tpr.
The bonded section might seems redundant but if one wants to use csg_stat without mapping, he still has to write 1:1 mapping with cg_bonded part to remove bonded terms from rdf.

How is the status?

It's still on my todo list

I think that would be a cool feature to have, so let me know if there is some, even limited, version to merge.

Is this ready to merge?

Huh what a mess with all those commits from merge...
anyway, I noticed after looking on @symashayak data is that whenever  is used the molecule list is clear but not the bead list and second time beads are added.
(ref: https://groups.google.com/forum/#!topic/votca/vZonpPE7kG0)
I am not entirely sure if  should also clean bead list as the original functionality will be broken. Instead, if base="" is defined then it is not possible to define own beads.

You have rebase all commit in master, a merge would make every commit show up twice in the history, so I cannot merge this in its current state.
You could either use git rebase -i master and pick important commits related to the xmltopoloy reader. Or simpler use the patch from git diff HEAD..master as a single commit on a new branch.

And it is ok to break the original functionally if this makes the xml reader cleaner. We can always re-implement it later if needed.

and I'm back...
Tested:
csg_map --top topol.xml --trj spce.dump --out spce2.gro --no-map
and also with mapping file, it seems to be okey.
Maybe @symashayak could test on some trajectory from lammps;

Looks good to me! @symashayak can you test that?

Hi,
It seems to have worked for me too. Especially csg_dump output seems to be
correct.
But the water-water RDF I get with csg_stat with mapping has some errors.
This could be due to some error I may have done generating LAMMPS
trajectory.
So I am debugging that issue. Will update as soon as I am done testing.
Thanks,
Sikandar
On Fri, Jul 15, 2016 at 11:04 AM, Christoph Junghans <
<EMAIL_ADDRESS>wrote:

Looks good to me! @symashayak https://github.com/symashayak can you
test that?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/votca/csg/pull/176#issuecomment-232994255, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADu1NoK3_Q4LHAwaIddPLUw9wDWZ9Sd2ks5qV6-BgaJpZM4GnFH_
.

It works for me too! RDF from csg_stat with mapping seems correct.
On Fri, Jul 15, 2016 at 10:13 AM, Sikandar Mashayak<EMAIL_ADDRESS>wrote:

Hi,
It seems to have worked for me too. Especially csg_dump output seems to be
correct.
But the water-water RDF I get with csg_stat with mapping has some errors.
This could be due to some error I may have done generating LAMMPS
trajectory.
So I am debugging that issue. Will update as soon as I am done testing.
Thanks,
Sikandar
On Fri, Jul 15, 2016 at 11:04 AM, Christoph Junghans <
<EMAIL_ADDRESS>wrote:

Looks good to me! @symashayak https://github.com/symashayak can you
test that?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/votca/csg/pull/176#issuecomment-232994255, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ADu1NoK3_Q4LHAwaIddPLUw9wDWZ9Sd2ks5qV6-BgaJpZM4GnFH_
.


Mismatched SHA256 fingerprint on Windows ccache binary

It seems that the stored fingerprint for the Windows ccache binary does not match. As of today, I'm receiving the following error during the Install ccache phase of the action on Windows runners only:
Error: SHA256 of C:\Users\runneradmin\.cargo\bin\ccache.exe is d61defbaabdb75a9bb61b53b08a35bb302bdd938923e3ae83e08ae34a73e7166, expected 6e0959c518b0c51fa887035465e7486f50079fe9acf711e5dd7ea2d30a824521

I did not change the workflow file at all and it was working perfectly fine the other day. For an example run, see here.
Many thanks for your help!

fixed the checksum and released 1.2.5


Stitching of images with overlap

Let's use this task to discuss stitching of images with overlap, see modality 2 in https://github.com/fractal-analytics-platform/fractal-tasks-core/issues/11#issuecomment-1261838969

As mentioned in #10 , I have added a dataset for testing this with 50 pixel overlap. Let's discuss here which ways we could best approach this challenge.

After discussions with Clara about this topic, I think stitching can be in the backlog for the moment. It's unclear whether she will continue to image with overlap in her project. Thus, until this is clearer or someone else is pushing for actual stitching, I'd say we keep this issue on hold.

We are not planning to cover this in fractal-tasks-core. The multi-view stitcher could be a good solution for this: https://github.com/multiview-stitcher/multiview-stitcher


The Loss calculation is always 0

According to the paper, the loss of the reference image needs to be calculated in full, and the loss of the target image only calculates the area that is not masked. In the program, all weighting values are 0. In fact, weights need to be all 1 in the reference image and 1 in the target image where mask=0.

I admit that the code is a bit weird but eighting is correct and you can take a look at #12 to see more in detail about the weighting


Swarm 1.2.0 potentially making incorrect call to Consul API for creating sessions

I attempted to upgrade to Swarm 1.2.0 tonight, but failed because both Swarm manager and Consul are constantly dumping messages like the following:
Apr 20 19:09:28 dockermgmt.ad.fhda.edu consul[13105]: 2016/04/20 19:09:28 [ERR] http: Request PUT /v1/session/create, error: rpc error: Invalid Session TTL '7500000000', must be between [10s=24h0m0s] from=<IP_ADDRESS>:39249
Apr 20 19:09:38 dockermgmt.ad.fhda.edu consul[13105]: 2016/04/20 19:09:38 [ERR] http: Request PUT /v1/session/create, error: rpc error: Invalid Session TTL '7500000000', must be between [10s=24h0m0s] from=<IP_ADDRESS>:39249
Apr 20 19:09:48 dockermgmt.ad.fhda.edu consul[13105]: 2016/04/20 19:09:48 [ERR] http: Request PUT /v1/session/create, error: rpc error: Invalid Session TTL '7500000000', must be between [10s=24h0m0s] from=<IP_ADDRESS>:39249
Apr 20 19:09:58 dockermgmt.ad.fhda.edu consul[13105]: 2016/04/20 19:09:58 [ERR] http: Request PUT /v1/session/create, error: rpc error: Invalid Session TTL '7500000000', must be between [10s=24h0m0s] from=<IP_ADDRESS>:39249
Apr 20 19:10:08 dockermgmt.ad.fhda.edu consul[13105]: 2016/04/20 19:10:08 [ERR] http: Request PUT /v1/session/create, error: rpc error: Invalid Session TTL '7500000000', must be between [10s=24h0m0s] from=<IP_ADDRESS>:39249
Apr 20 19:10:18 dockermgmt.ad.fhda.edu consul[13105]: 2016/04/20 19:10:18 [ERR] http: Request PUT /v1/session/create, error: rpc error: Invalid Session TTL '7500000000', must be between [10s=24h0m0s] from=<IP_ADDRESS>:39249
Apr 20 19:10:28 dockermgmt.ad.fhda.edu consul[13105]: 2016/04/20 19:10:28 [ERR] http: Request PUT /v1/session/create, error: rpc error: Invalid Session TTL '7500000000', must be between [10s=24h0m0s] from=<IP_ADDRESS>:39249

I downgraded back to 1.1.3, and then I could control the cluster again.

Hi @mrapczynski, so the default LockTTL should be at 20s now which should work by default for all configs on every store. Are you using the --replication-ttl flag at a value of 15s by any hazard? Otherwise to circumvent that, you can use a config.json on Consul startup with:
{"session_ttl_min": "5s"}

And this will solve the issue.

@abronan Yes incidentally it is 15s! How did you know? Does that drive a calculated value elsewhere?
I'll try the Consul config in the meantime - easy change.

@mrapczynski Yes :smile: Consul divides the TTL by 2x and does not support TTLs < 10s by default. So an acceptable default value for Consul is 20s minimum for --heartbeat and --replication-ttl, otherwise you have to specify the session_ttl_min on Consul config (we have to do that in the CI to speed up tests).
I'm glad you made it all work! Closing this one but feel free to open a new issue if you have any more questions :)


Micro cannot open files with colon in their name

Description of the problem or steps to reproduce
Many log files I deal with are in the zulu time format, like something-2024-01-15T19:03:33Z.log. These are valid UNIX filenames. Micro is unable to open these files. Instead, it tries to open a non-existent file whose name ends before the first colon in the filename (in this example, something-2024-01-15T19). I don't see any mechanism to escape this behavior.
I've seen a couple of tickets referencing this, but they all seem to be closed, with even partial workaround alluded to but not spelled out in the ticket. This is still an issue as of the current version of Micro.
Specifications

Version: 2.0.13
Commit hash: Unknown
Compiled on Unknown*
OS: Void Linux (rolling)
Terminal: Alacritty 0.13.1
*-- Built from my distro. We don't seem to be doing anything special: https://github.com/void-linux/void-packages/blob/78bf2e3347a460dd3968e3655120a7aea5d51ae4/srcpkgs/micro/template

For me it's working:

Usually the : character needs to be escaped at the consoled via \: or the whole filename encapsulated with "":
micro something-2024-01-15T19\:03\:33Z.log
micro "something-2024-01-15T19:03:33Z.log"
To be honest...from my point of view this isn't a problem of micro.

This is not a problem of shell escaping the colon: https://asciinema.org/a/MojjVMqT9g4yxmO87ekrbiLq2

Does cat or less behave the same in your zsh?

Does cat or less behave the same in your zsh?

It does not. This is not a shell issue. I can use Bash or sh and get the same behavior. I can write a Python subprocess.run wrapper to call Micro with the arguments passed directly. All of these have the same behavior.

Ok, then let me guess:
You've the parsecursor option enabled in your options.

parsecursor: if enabled, this will cause micro to parse filenames such as file.txt:10:5 as requesting to open file.txt with the cursor at line 10 and column 5. The column number can also be dropped to open the file at a given line and column 0. Note that with this option enabled it is not possible to open a file such as file.txt:10:5, where :10:5 is part of the filename. It is also possible to open a file with a certain cursor location by using the +LINE:COL flag syntax. See micro -help for the command line options.

default value: false

This option now is in conflict with your or your service chosen filenames.
Ok, this then can only be fixed by breaking the combination of [FILE]:LINE:COL (e.g. file.txt:10:5 wouldn't be supported any longer).

Thank you. It's not obvious that this option is available, (default?) and interferes with the normal course of opening files. That should probably be mentioned somewhere conspicuous.
It also seems like the parsing could be smarter. My files did not end with a colon and a line number. They happened to contain a colon followed by many other characters that couldn't possibly be construed as a line/column indicator. Why doesn't the parser bail out in this case?
Usability wise it seems the +LINE:COL should be the only mechanism available. But if Micro is set on using :line:col suffixes on filenames (maybe it's for the case of opening more than 1 file?), then the parsing should probably look for an exact pattern.
I'm satisfied for now. I've just disabled the option.

The most user wouldn't have a problem with that scenario due to the fact that it isn't the default option, but yes this option isn't intended to fail in this scenario...especially since the documentation every time refers to [FILENAME]:LINE:COL usage.

It also seems like the parsing could be smarter. My files did not end with a colon and a line number.

And it's now exactly fixed to search for that option at the end or more detailed after the filename.

You can enable parsecursor again after the PR has been merged or you fix it locally with a patch. :wink:


combining regressions

Do the regression lines for different site_id's outside split up into groups, qualitatively?
Mathematical question: How to combine the many regressions for each site_id?

How to combine the many regressions for each site_id?

Either of the following would provide you with a per-site_id set of parameters

fit the model for each site_id, estimate the parameters
fit site_id as a factor in a single model and combine


Update to v3

Update to support Mailgun v3

I’ll implement it soon. Thanks for your interest.

8fdc9a0c06f1551df1d95be7a7acfc3dac2d21df


⚠ Bofrost.lu has degraded performance

In 04220eb, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1749 ms

Resolved: Bofrost.lu performance has improved in 90e00e7.


Create Python3-Tower_of_Hanoi.py

I've followed the rules given in README.md
It would be great if you could review and merge ASAP


Does Helia support connecting to remote infura node ?

I cannot seem to find any documentation regarding trying to connect to a infura node with helia. Is that supported?

If you are trying to dial the multiaddr of an infura IPFS node, then yes, it is supported.
If you are trying to use Helia where you previously used ipfs-http-client then no, you should use kubo-rpc-client instead.
Can you please share more details of what you are trying to do?


updating test cases and fixing existing bastion bug

Description
Fixing existing bastion login vsi issue
Adding test cases
Release required?

[ ] No release
[x] Patch release (x.x.X)
[ ] Minor release (x.X.x)
[ ] Major release (X.x.x)

Release notes content

Run the pipeline
If the CI pipeline doesn't run when you create the PR, the PR requires a user with GitHub collaborators access to run the pipeline.
Run the CI pipeline when the PR is ready for review and you expect tests to pass. Add a comment to the PR with the following text:
/run pipeline

Checklist for reviewers

[ ] If relevant, a test for the change is included or updated with this PR.
[ ] If relevant, documentation for the change is included or updated with this PR.

For mergers

Use a conventional commit message to set the release level. Follow the guidelines.
Include information that users need to know about the PR in the commit message. The commit message becomes part of the GitHub release notes.
Use the Squash and merge option.

/run pipeline

:tada: This issue has been resolved in version 1.7.2 :tada:
The release is available on:

GitHub release
v1.7.2

Your semantic-release bot :package::rocket: