core_kernel.v0.14.1+multicore fails to install (because of ocaml-migrate-parsetree)

Hi, I'm eager to try the awesome ocaml-multicore and to that purpose I'm trying to setup a Core_kernel-based environment for it on macOS 11.0.1 Big Sur.
I followed the instructions at https://github.com/ocaml-multicore/multicore-opam#install-multicore-ocaml
and created a new opam switch with
$ opam switch create 4.10.0+multicore --packages=ocaml-variants.4.10.0+multicore --repositories=multicore=git+https://github.com/ocaml-multicore/multicore-opam.git,default

Unfortunately, I get the following error message:

[ERROR] The compilation of ocaml-migrate-parsetree failed at "/Users/anton/.opam/opam-init/hooks/sandbox.sh build dune build -p ocaml-migrate-parsetree -j 3".

However, PR https://github.com/ocaml-multicore/multicore-opam/pull/42 says it should work (/cc @kit-ty-kate ). Am I doing something wrong?
Here is the full output, just in case:
The following actions will be performed:
  ∗ install ocaml-migrate-parsetree 2.1.0             [required by ppxlib]
  ∗ install ppxlib                  0.17.0            [required by ppx_jane]
  ∗ install ppx_variants_conv       v0.14.1           [required by ppx_jane]
  ∗ install ppx_typerep_conv        v0.14.1           [required by ppx_jane]
  ∗ install ppx_stable              v0.14.1           [required by ppx_jane]
  ∗ install ppx_sexp_conv           v0.14.1           [required by core_kernel]
  ∗ install ppx_pipebang            v0.14.0           [required by ppx_jane]
  ∗ install ppx_optional            v0.14.0           [required by ppx_jane]
  ∗ install ppx_optcomp             v0.14.0           [required by ppx_jane]
  ∗ install ppx_let                 v0.14.0           [required by ppx_jane]
  ∗ install ppx_js_style            v0.14.0           [required by ppx_base]
  ∗ install ppx_here                v0.14.0           [required by ppx_jane]
  ∗ install ppx_fixed_literal       v0.14.0           [required by ppx_jane]
  ∗ install ppx_fields_conv         v0.14.1           [required by ppx_jane]
  ∗ install ppx_enumerate           v0.14.0           [required by ppx_base]
  ∗ install ppx_compare             v0.14.0           [required by ppx_base, bin_prot]
  ∗ install ppx_cold                v0.14.0           [required by ppx_base]
  ∗ install ppx_custom_printf       v0.14.0           [required by ppx_jane]
  ∗ install ppx_sexp_value          v0.14.0           [required by ppx_jane]
  ∗ install ppx_sexp_message        v0.14.0           [required by core_kernel]
  ∗ install ppx_hash                v0.14.0           [required by core_kernel]
  ∗ install ppx_assert              v0.14.0           [required by core_kernel]
  ∗ install bin_prot                v0.14.0           [required by core_kernel]
  ∗ install ppx_base                v0.14.0           [required by core_kernel]
  ∗ install jst-config              v0.14.0           [required by core_kernel]
  ∗ install ppx_bin_prot            v0.14.0           [required by ppx_jane]
  ∗ install ppx_string              v0.14.1           [required by ppx_jane]
  ∗ install time_now                v0.14.0           [required by core_kernel]
  ∗ install ppx_module_timer        v0.14.0           [required by ppx_jane]
  ∗ install ppx_inline_test         v0.14.1           [required by core_kernel]
  ∗ install ppx_expect              v0.14.0           [required by ppx_jane]
  ∗ install ppx_bench               v0.14.1           [required by ppx_jane]
  ∗ install splittable_random       v0.14.0           [required by core_kernel]
  ∗ install base_quickcheck         v0.14.0           [required by core_kernel]
  ∗ install ppx_jane                v0.14.0           [required by core_kernel]
  ∗ install base_bigstring          v0.14.0           [required by core_kernel]
  ∗ install core_kernel             v0.14.1+multicore
===== ∗ 37 =====
Do you want to continue? [Y/n] y

<><> Gathering sources ><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
[base_bigstring.v0.14.0] found in cache
[base_quickcheck.v0.14.0] found in cache
[bin_prot.v0.14.0] found in cache
[jst-config.v0.14.0] found in cache
[ocaml-migrate-parsetree.2.1.0] found in cache
[ppx_assert.v0.14.0] found in cache
[ppx_base.v0.14.0] found in cache
[ppx_bench.v0.14.1] found in cache
[ppx_bin_prot.v0.14.0] found in cache
[ppx_cold.v0.14.0] found in cache
[ppx_compare.v0.14.0] found in cache
[ppx_custom_printf.v0.14.0] found in cache
[ppx_enumerate.v0.14.0] found in cache
[ppx_expect.v0.14.0] found in cache
[ppx_fields_conv.v0.14.1] found in cache
[ppx_fixed_literal.v0.14.0] found in cache
[ppx_hash.v0.14.0] found in cache
[ppx_here.v0.14.0] found in cache
[ppx_inline_test.v0.14.1] found in cache
[ppx_jane.v0.14.0] found in cache
[ppx_js_style.v0.14.0] found in cache
[ppx_let.v0.14.0] found in cache
[ppx_module_timer.v0.14.0] found in cache
[ppx_optcomp.v0.14.0] found in cache
[ppx_optional.v0.14.0] found in cache
[ppx_pipebang.v0.14.0] found in cache
[ppx_sexp_conv.v0.14.1] found in cache
[ppx_sexp_message.v0.14.0] found in cache
[ppx_sexp_value.v0.14.0] found in cache
[ppx_stable.v0.14.1] found in cache
[ppx_string.v0.14.1] found in cache
[ppx_typerep_conv.v0.14.1] found in cache
[ppx_variants_conv.v0.14.1] found in cache
[ppxlib.0.17.0] found in cache
[splittable_random.v0.14.0] found in cache
[time_now.v0.14.0] found in cache
[core_kernel.v0.14.1+multicore] downloaded from https://github.com/kit-ty-kate/core_kernel/archive/multicore.tar.gz

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
[ERROR] The compilation of ocaml-migrate-parsetree failed at "/Users/anton/.opam/opam-init/hooks/sandbox.sh build dune build -p ocaml-migrate-parsetree -j 3".

#=== ERROR while compiling ocaml-migrate-parsetree.2.1.0 ======================#
# context     2.0.7 | macos/x86_64 | ocaml-variants.4.10.0+multicore | https://opam.ocaml.org#147b2732
# path        ~/.opam/4.10.0+multicore/.opam-switch/build/ocaml-migrate-parsetree.2.1.0
# command     ~/.opam/opam-init/hooks/sandbox.sh build dune build -p ocaml-migrate-parsetree -j 3
# exit-code   1
# env-file    ~/.opam/log/ocaml-migrate-parsetree-38137-0edcf3.env
# output-file ~/.opam/log/ocaml-migrate-parsetree-38137-0edcf3.out
### output ###
#       ocamlc src/.migrate_parsetree.objs/byte/migrate_parsetree__Ast_410.{cmi,cmo,cmt} (exit 2)
# (cd _build/default && /Users/anton/.opam/4.10.0+multicore/bin/ocamlc.opt -w -40 -g -bin-annot -I src/.migrate_parsetree.objs/byte -I /Users/anton/.opam/4.10.0+multicore/lib/ocaml/compiler-libs -no-alias-deps -open Migrate_parsetree__ -o src/.migrate_parsetree.objs/byte/migrate_parsetree__Ast_410.cmo -c -impl src/ast_410.pp.ml)
# File "src/ast_410.ml", lines 240-295, characters 2-44:
# Error: This variant or record definition does not match that of type
#          Parsetree.pattern_desc
#        Constructors number 17 have different names, Ppat_effect and Ppat_extension.

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
┌─ The following actions failed
│ λ build ocaml-migrate-parsetree 2.1.0
└─
╶─ No changes have been performed

you need to install 4.10.0+multicore+no-effect-syntax which is here to avoid having issues with ppxs as the main branch (+multicore) changes the parsetree and requires to change pretty much everything (it will be done but only during the OCaml 5.0 alpha phase as the interface needs to be more set in stone to start working on it)

That did the trick, thanks so much @kit-ty-kate!
In case someone runs into this issue, I'll leave the exact commands that worked for me:
$ opam switch create 4.10.0+multicore+no-effect --packages=ocaml-variants.4.10.0+multicore+no-effect-syntax --repositories=multicore=git+https://github.com/ocaml-multicore/multicore-opam.git,default
$ opam install core_kernel.v0.14.1+multicore


Support for blockcahin-h-plugin and custom C++ hooked functions

See linked issue

This PR includes:

corrections to the top-level Makefile that build the semantics with correct support for hooked functions
Modification to kavm kompile that implements a two-stage kompilation process described in this issue. Building the LLVM interpreter for the semantics this way ensures the hooks are working as intended.
Adds a standalone K mini project to test the custom and plugin hooks in an isolated environment. See ./tests/hooks.


Algorithms are represented by NamedTuples

In this PR I'm exploring a new way for users to interface with the code. To use the hmc algorithm we currently have to write:
import blackjax.hmc as hmc

state = hmc.new_state(logprob_fn)
kernel = hmc.kernel(logprob_fn, **params)

Which not only requires to pass the logprob_fn as an argument  twice (not a big problem), but also makes the internal organization less flexible. For instance we would like to have all HMC-related algorithms in the same module instead of needing one module per algorithm. It also makes it obvious that users need to first build a state from the position and pass said state to the kernel.
After this PR we would have:
import blackjax

hmc = blackjax.hmc(logprob_fn, **params)
state = hmc.init(position)
new_state, info = hmc.update(rng_key, state)

Closing #133.
Progress

[x]  Add a base.py file that describes the types
[ ] Update the design of:

[x] The HMC kernel
[x] The NUTS kernel
[ ] The SMC kernels
[x] The random walk kernel

[x] update the notebooks:

[x] Intrduction
[x] BNN
[x] Linear Regression
[ ] Tempered SMC
[ ] TFP
[ ] Numpyro
[ ] PyMC

[ ] Group NUTS and HMC in same file

I have successfully added the base interface types and updated HMC, NUTS and random walk RMH. One would now use HMC in the following way:
import blackjax

hmc = blackjax.hmc(logprob_fn, step_size, inverse_mass_matrix, num_integration_steps)
state = hmc.init(position)
new_state, info = hmc.update(rng_key, state)

There is an important issue with this design when we use adaptation algorithms to get values for the algorithms' parameters. Consider the NUTS algorithm (see HierarchicalBNN notebook):
import blackjax

initial_state = nuts.new_state(initial_position, logprob_fn)

kernel_generator = lambda step_size, inverse_mass_matrix: nuts.kernel(
    logprob_fn, step_size, inverse_mass_matrix
 )

final_state, (step_size, inverse_mass_matrix), info = stan_warmup.run(
    warmup_key, kernel_generator, initial_state, num_warmup
)

nuts_kernel = jax.jit(nuts.kernel(logprob_fn, step_size, inverse_mass_matrix))

While we currently only need logprob_fn to build the initial state, the new design also requires the value of other parameters: the step size, the inverse mass matrix, etc. Most of the time, however, we do not want to have to specify these values and use adaptation algorithms instead. This is not practical.

@junpenglao Part of this may be a bit ouf of scope for the current PR, but I think the following higher-level interface would be nice, and closer to Optax's :
import blackjax

# Option 1: The most flexible. We only specify `logprob_fn` at init but need to pass the value of parameters at every step.
nuts = blackjax.nuts(logprob_fn)
state = nuts.init(position)
new_state, info  = nuts.update(rng_key, state, step_size, inverse_mass_matrix)

# Option 2: We specify the algorithm parameters at init, either manually or using the window adaptation.
# This would be the most used interface
warmup = blackjax.window_adptation(blackjax.nuts, logprob_fn)
(step_size, inverse_mass_matrix), state, info = warmup.run(rng_key, state)

nuts = blackjax.nuts(logprob_fn, step_size, inverse_mass_matrix)
state = nuts.init(position)
new_state, info  = nuts.update(rng_key, state)

We do not support intermediate solutions. Internals would look like:
from typing import Callable, NamedTuple

class SamplingAlgorithm(NameTuple):
    init: Callable
    update: Callable

def nuts(logprob_fn: Callable, step_size: Optional, inverse_mass_matrix: Optional):
    if step_size and inverse_mass_matrix:
        return lambda key, state: kernel(rng_key, state, inverse_mass_matrix, step_size)
    elif not step_size and not inverse_mass_matrix:
        return kernel
    else:
        raise AttributeError("Blackjax allows you to build algorithm specifying either all parameters at initialization or none, but specifying only one parameter is not supported.")

def kernel(rng_key, state, inverse_mass_matrix, step_size):
    pass

Option 1 above is out of scope for this PR so I implemented a workaround for now:
import blackjax

# Option 1: The most flexible. We only specify `logprob_fn` at init but need to pass the value of parameters at every step.
nuts = blackjax.nuts(logprob_fn)
state = nuts.init(position)
new_state, info  = nuts.kernel(step_size, inverse_mass_matrix)(rng_key, state)

# Option 2: We specify the algorithm parameters at init, either manually or using the window adaptation.
# This would be the most used interface
nuts = blackjax.nuts(logprob_fn, step_size, inverse_mass_matrix)
state = nuts.init(position)
new_state, info  = nuts.step(rng_key, state)

I see, so the internal call pattern is always
... = kernel(rng_key, state, inverse_mass_matrix, step_size)

and we provide another api to generate closure around it.

That's correct, I think that internally we should use this general kernel(rng_key, state, **parameters) pattern, and provide a high-level API (purely for convenience) that specifies these values when the algorithm is initialized. What do you think?

Yeah I am 100% onboard with the general kernel(rng_key, state, **parameters) pattern.
Plus it is not that difficult to have high level api handles intermedia, maybe we can even use the new case pattern https://www.python.org/dev/peps/pep-0634/

I agree, although I'm not sure that we should do it event if it is simple. Sticking to a strict batteries-included vs you're on your own philosophy will save us all many headaches I believe.

@junpenglao I implemented the new interface. nuts(logprob_fn, **parameters) returns a namedtuple with an init and a step function. nuts_kernel return a more general kernel(rng_key, state, logprob_fn, step_size, inverse_mass_matrix) kernel. Not sure how to interface this with the warmup yet, but it would be great if you could have a look at it already.

I've made the modifications we discussed. The high level kernels are in blackjax/mcmc.py and you can now use blackjax the following way:
import jax
import jax.numpy as jnp
import jax.scipy.stats as stats
import numpy as np

import blackjax

rng_key = jax.random.PRNGKey(0)

observed = np.random.normal(10, 20, size=1_000)
def logprob_fn(x):
  logpdf = stats.norm.logpdf(observed, x["loc"], x["scale"])
  return jnp.sum(logpdf)

Use the HMC kernel:
step_size = 1e-3
inverse_mass_matrix = jnp.array([1., 1.])
num_integration_steps = 10

hmc = blackjax.hmc(logprob_fn, step_size, inverse_mass_matrix, num_integration_steps)

initial_position = {"loc": 1., "scale": 2.}
state = hmc.init(initial_position)
new_state, info = hmc.step(rng_key, state)

Use the NUTS kernel:
nuts = blackjax.nuts(logprob_fn, step_size, inverse_mass_matrix)
state = nuts.init(initial_position)
new_state, info = nuts.step(rng_key, state)

And the gaussian random walk:
sigma = jnp.array([0.1, 0.1])

rmh = blackjax.rmh(logprob_fn, sigma)
state = rmh.init(initial_position)
new_state, info = rmh.step(rng_key, state)

I will now update the adaptation API (in blackjax/adaptation.py), and the SMC API (in blackjax/smc.py).

Here is the adaptation API we're aiming for:
warmup = blackjax.stan_warmup(algorithm, logprob_fn, *args)
new_state, (step_size, inverse_mass_matrix), info = warmup.run(rng_key, state, num_steps)

Where args is such that we can later build a new kernel with:
algo = algorithm(logprob_fn, step_size, inverse_mass_matrix, *args)

Suggestion:
warmup = blackjax.stan_warmup(algorithm, logprob_fn, ***args)
new_state, updated_args, info = warmup.run(rng_key, state, num_steps)
algo = algorithm(logprob_fn, ***updated_args)

What would you think of the following structure:
blackjax/
  adaptation.py  # aliases
  mcmc.py  # aliases
  smc.py  # aliases
  kernel/  # raw kernels
     hmc.py
     nuts.py
     tempered_smc.py
     adaptive_tempered_smc.py
     orbital_mcmc.py
     mala.py
     ula.py
     sgld.py
  internals/  # all the building blocks
    adaptation/
        window_adaptation.py
     hmc/
        integrator.py
        trajectory.py
     smc/
     sgmcmc/

I don't really like having an internals/ directory. I'd rather have the directory hmc/ and have the public kernels live in __init__.py

Maybe this structure fits the problem better:
blackjax/
  alias.py  # high level aliases
  adaptation/
     step_size.py
     mass_matrix.py
     window_adaptation.py
  smc/
     smc.py
     ess_solver.py
     solver.py
  mcmc/
     hmc.py
     nuts.py
     tempered_smc.py
     adaptive_tempered_smc.py
     orbital_mcmc.py
     mala.py
     ula.py
     sgld.py
     trajectory.py
     integrator.py
     metrics.py

That looks better, though I don't think it's so bad to have HMC specific things in their own directory.

I've found that the distinction between hmc, langevin, random_walk etc is not that relevant actually. They only differ in the way they build a proposal, and some algorithms mix different methods.

Last round of review @junpenglao !


Ublox MAX-M10: Getting  errror -5  for uGnssInfoGetVersions and -2 for uGnssPosGet

Hi there,
I am trying to get nrf52840 to communicate with the Ublox module via I2C.
The device tree is configure as follows:
/{
	gps {
		status = "okay";
		compatible = "u-blox,ubxlib-device-gnss";
		transport-type = "i2c0";
		i2c-already-open;
		i2c-address = <0x42>;
		i2c-max-segment-size = <256>;
		module-type = "U_GNSS_MODULE_TYPE_M10";

	};
};

&i2c0 {
	status = "okay";
	pinctrl-0 = <&i2c0_default>;
	pinctrl-1 = <&i2c0_sleep>;
	pinctrl-names = "default", "sleep";
	compatible = "nordic,nrf-twim";
};

Running the following in the main:
int main()
{
    // Toggle the GPS reset pin
    gpio_pin_configure_dt(&gpsExtInt, GPIO_OUTPUT_INACTIVE);
    k_sleep(K_MSEC(10));
    gpio_pin_set_dt(&gpsExtInt, 0);
    k_sleep(K_MSEC(500));
    uDeviceHandle_t ubx_device_handle = NULL;
    int32_t latitudeX1e7;
    int32_t longitudeX1e7;

    int err;
    if (0 != (err = uPortInit())) {
        LOG_ERR("uPortInit failed %d", err);
    }

    if (0 != (err = uPortI2cInit())) {
        LOG_ERR("uPortI2cInit failed %d", err);

    }

    if (0 != (err = uDeviceInit())) {
        LOG_ERR("uDeviceInit failed %d", err);

    }

    if (0 != (err = uDeviceOpen(NULL, &ubx_device_handle))) {
        LOG_ERR("uDeviceOpen failed %d", err);
    }

    uGnssVersionType_t version;

    if (0 == (err = uGnssInfoGetVersions(&ubx_device_handle, &version))) {
        LOG_INF("GNSS ver: %s", version.ver);
        LOG_INF("GNSS fw: %s", version.fw);
        LOG_INF("GNSS hw: %s", version.hw);
        LOG_INF("GNSS prot: %s", version.prot);
        LOG_INF("GNSS mod: %s", version.mod);
    }
    else
    {
        LOG_ERR("uGnssInfoGetVersions %d", err);
    }

    if (0  == (err = uGnssPosGet(&ubx_device_handle, &latitudeX1e7, &longitudeX1e7,
                    NULL, NULL, NULL, NULL, NULL, NULL))) {
        printf("I am here: https://maps.google.com/?q=%3.7f,%3.7f\n",
                ((double) latitudeX1e7) / 10000000,
                ((double) longitudeX1e7) / 10000000);
    }
    else {
         LOG_ERR("uGnssPosGet %d", err);
    }
	return 0;

}

The log outcome:
*** Booting nRF Connect SDK 3758bcbfa5cd ***
U_PORT_BOARD_CFG: using GNSS device "gps" from the device tree, module-type 2 with pin-enable-power -1 (0xffffffff), pin-data-ready -1 (0xffffffff)...
U_PORT_BOARD_CFG: ...GNSS on I2C 0, i2c-address 0x42, i2c-clock-hertz 100000, i2c-max-segment-size 256, i2c-already-open.
U_GNSS: initialising with ENABLE_POWER pin not connected, transport type I2C.
U_GNSS: sent command b5 62 0a 06 00 00 10 3a.

U_GNSS: sent command b5 62 06 04 04 00 00 00 09 00 17 76.
U_GNSS: sent command b5 62 0a 06 00 00 10 3a.
 Was the GNSS initialised successful? It responded with 1C

Hi there, sorry you're having trouble with this and thanks for posting.  Looking at the code, it might be as simple as you need this:
uGnssInfoGetVersions(ubx_device_handle, &version);

...rather than this:
uGnssInfoGetVersions(&ubx_device_handle, &version);

Thank you @RobMeades . That resolve the problem


Generates invalid paths when properties have special characters like dots and quotes

Given a json with special characters like this:
{
  "data": {
    "a.\"property\".here 大😋": {
      "/some\\one's/prop/":  [123]
    }
  }
}

The "copy-json-path" extension generates/copies this path:
data.a."property".here 大😋./some\one's/prop/[0] which is invalid.
Expectation is to use enclosing square-brakets and quotes when ["some.prop.name"]
and also escape double quotes.
The valid json path for the example should be:
x.data['a."property".here 大😋']['/some\\one\'s/prop/'][0]
or
x.data["a.\"property\".here 大😋"]["/some\\one's/prop/"][0]
depending on your preference towards using single or double quotes.

Also for convenience, if the property doesn't have special characters then don't use the square brakets, only use them if they are needed.

In the copy.command.ts there should be a check to see if the property name "val" is a valid javascript identifier (see some examples like this)

This library could help to check if square brakets and quotes are needed
https://github.com/shinnn/is-var-name

I think this would be it:
copy.command.ts:
// ...
import isVarName from 'is-var-name';
// ...
if (Number.isInteger(val)) {
  return index === 0 ? `[${val}]` : `${acc}[${val}]`;
} else {
  const isValidJsIdentifier = isVarName(val);
  if (isValidJsIdentifier) {
	var valEscaped = val.replace(/'"'/g, '\\"').replace(/\\/g, '\\\\');;
	var valPropertyPath = `["${valEscaped}"]`;
	return index === 0 ? valPropertyPath : (acc + valPropertyPath);
  } else {
	return index === 0 ? val : `${acc}.${val}`;
  }
}

For a full list of valid characters that doesn't need escaping,
I made a test here
that iterates through all unicode characters to build a list of all the valid ones, and then list it down in different ways: an array of characterIndex, array of chars, array of character ranges for RegExp...
It could be used for another PR to support escaping property names with quotes only when really needed.


Grain Placement Documentation

https://github.com/dotnet/docs/blob/main/docs/orleans/grains/grain-placement.md
--
Add a "Choosing a Placement Strategy" section to the above document that helps developers understand and evaluate which placement strategy they should choose.
Questions that came to mind:

Why does Orleans default to Random v.s. Activation-count-based placement?
Why not default to Activation-count-based placement?
Pros / Cons Risk of each strategy?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: ceea5435-b56c-9d1f-afbb-ec7371b78493
Version Independent ID: 8769d3fe-dd2e-6096-d927-214d01c7428b
Content: Grain placement
Content Source: docs/orleans/grains/grain-placement.md
Product: dotnet-orleans
GitHub Login: @IEvangelist
Microsoft Alias: dapine

Associated WorkItem - 62375

@ReubenBond and @benjaminpetit do either of you have insights on this one?


 IF YOU DO NOT FOLLOW THE FORMAT OR MAKE A GENERIC TOKEN REQUEST YOUR ISSUE WILL BE DELETED WITHOUT COMMENT
YOUR JUSTLIST MUST FOLLOW THE JSON SPECIFICATION
https://github.com/justswaporg/justlists/blob/main/example.justlists.ts
Checklist

[x] I understand that this is not the place to request a token listing.
[x] I have tested that my JustList is compatible by pasting the URL into the add a list UI at justswap.org.
[x] I understand that filing an issue or adding liquidity does not guarantee addition to the justlists website.

Please provide the following information for your token.
JustList URL must be HTTPS.
JustList URL: https://adscoin.co/
JustList Name: ADSCOIN
Link to the official homepage of the JustList manager:

Sorry, your issue will be closed as you did not submit your information in the correct format.Please resubmit.

Sorry, this issue will be closed . If you want to add a new list, please resubmit.


How to reduce UI lag

Hi
I have deployed a simple Justpy app to Azure App Service. Locally it works great, but in Azure there is significant lag when using input fields. If I type something into a field, characters get erased, selections get reverted etc. I suspect this is due to network lag. The clicked UI components also get this red thin border around them which doesn't happen on local host.
I had to revert to AJAX as the App Service websocket support doesn't seem to work with Justpy.
My app is an Eventhubs test data generator and it uses Justpy functionality to start a background process to generate events. As said, it works beautifully on my local machine.
Are there any tweaks to disable some AJAX functionality to diminish some of the lag? I tried setting "debounce" to 3000 on all input components, but it didn't really help.

Hi,
I am not familiar with Azure App Service.
Could you please check your browser's dev tools for the network timings for the communication?
Is there an option on Azure to configure a simple Linux instance and use it? This is what I do using Digital Ocean.
Also, to simulate latency on the local host you can set the LATENCY variable in justpy.env
https://justpy.io/tutorial/configuration/
Please let me know how things work out for you.

I had to jump to another task, but I will test this later. I could deploy this to a dedicated VM like you say, but Azure App Services offers some useful features like integrated authentication etc.
In general, do you think network latency can cause weird behaviour in Justpy javacsript components or is this just related to my code and the behaviour of the App Service web gateway?
I really like the Justpy approach, because I'm a database and backend engineer and just need to build some simple UIs from time to time.

It depends on the application, but for most applications with typical latency, there is no problem. The best way to find out is to set the LATENCY variable locally and see if it makes a difference.


Fix audit a11y: Change wrong images labels

Note: Please transform - [ ] into - (NA) in the description when things are not applicable
Related issues

#2667
Description

Change labels:

for Orange logos
for tooltip example with an SVG

Motivation & Context

Types of change

Bug fix (non-breaking which fixes an issue)

Live previews

https://deploy-preview-2709--boosted.netlify.app/docs/about/brand/
https://deploy-preview-2709--boosted.netlify.app/docs/components/tooltips/

The question is still running for me, but the logo is mainly used in the header and often used as a part of the brand name. For instance I'm thinking about Orange Cyberdefense, Orange Foundation, etc... Which title would then say Orange master logo Cyberdefense. My opinion is balancing between:

We are a documentation so everything isn't perfect but we ease our UX.
If people want to use the logo, they need to adapt the label and so we can have whatever in here.

We didn't talk about this: of course, in the context of a website, if the logo is in a links that leads to home page, as it often does, the label shouldn't be "XXX logo" but something like "XXX home". For this, I think that developers can be inspired by Boosted code itself, don't you think? In this example, we don't have any context, so it would be weird to use such a label...


Feature Request/Question: Security measures

Frameworks such as hanami now include up to date security practices such as CSP and X frame headers etc. by default, are there any plans to do so too for padrino?

👍

Do we need a special treatment from Padrino to set HTTP headers?

Padrino, being based on Sinatra, automatically includes rack-protection, which provides CSP and X-Frame headers by default.
https://github.com/sinatra/sinatra/tree/master/rack-protection


Parse Digest string with Regex

Hi Bernardo Esbérard,
Thank you for the library. However, we encountered issue while testing a facial recognition device that uses HTTP Digest Authentication. Firstly, someone already reported issue regarding the ordering of the digest items e.g. nonce, realm etc.. Secondly, there is an issue with the nonce value as during our test, we noticed the authentication was failing because the nonce value is encoded using base64 and it contains the equal (=) character.
https://github.com/bernardbr/RestSharp.Authenticators.Digest/blob/09b971b3be17475f48cb6f5cc36d36342e67e795/src/DigestAuthenticator/DigestAuthenticatorManager.cs#L189-L194

From the lines above, you can observe that splitting with equal symbol (=) will cause issue as nonce value might contain equal symbol.
Hence, to avoid the issue 1) and 2), we've decided to use regex to parse the digest string and extract the relevant parts. So far only, we only require realm, nonce, and qop, thus, only tested these.
Hope it helps, and will look forward for the new nuget version.
Thank you.

@codetronics Thanks for the contribution! I took advantage and implemented a test for regex validation.
Version 1.2.0 was released.

@bernardbr No problem, and thank you for the quick release.


Ignore zero-length bit-fields when flattening.

Fixes the zero-length bit-field question in issue #99.

lgtm

Looks good to me.

Two lgtms from the main interested parties, so I'm considering it approved and merging it.


readme needs to be updated

the images in the readme are not working fine, and readme can be improved by adding emojis and making better layout

You can go for it but still it is a minor documentation fix so I will label as level 1

You can go for it but still it is a minor documentation fix so I will label as level 1

@rajpratyush  can you tell me what to use for images that are not showing up in the readme?

@rajpratyush  can you tell me what to use for images that are not showing up in the readme?

actually the images hasnt been uploaded into the  images folder and that's why it is not showing up

actually the images hasnt been uploaded into the  images folder and that's why it is not showing up

okay can you assign to me

okay can you assign to me


Implement Update/Replace + fix nil pointer exception

Hi @oze4!
Thanks for creating this project!
Here is 3 commits that implements PATCH/PUT methods through Update/ReplaceXX and fix a nil pointer exception when the status code returned is invalid.
The code works as is, but feel free to ask me to make some cosmetic changes if you prefer.

Oh wow! Thank you SO much!
Feel free to make any changes, cosmetic or otherwise, as long as you teach me a thing or two ;) I'm intermediate at best with Go (ok, I'm a beginner) so I've got a lot to learn.
Cheers!
I'll take a look at your additions here in a bit but I'm going to merge now.

Oh wow! Thank you SO much!
Feel free to make any changes, cosmetic or otherwise, as long as you teach me a thing or two ;) I'm intermediate at best with Go (ok, I'm a beginner) so I've got a lot to learn.
Cheers!
I'll take a look at your additions here in a bit but I'm going to merge now.

@nemunaire I randomly thought about this. I'm curious to know what you would change (genuinely). I mainly want to know why X/Y/Z should be changed, as I'd like to learn from my mistakes.
Hope to hear from you soon! Cheers.


JSON5 vulnerability

Hi team,
Currently, we have the following vulnerability, can we look into fixing it?
[

  {
    "createdAt": "2023-02-02T17:50:38Z",
    "dismissedAt": null,
    "securityVulnerability": {
      "package": {
        "name": "json5"
      },
      "severity": "HIGH"
    }
  }
]

It is possible that https://github.com/peter-evans/find-comment/pull/97 fixed it, but we need a new release

Hi @yeikel
Released v2.2.1 / v2 with the fix.


feat: Update to non-eager FutureListener

Works on issue found in #1755

Running java-compute ITs locally with this build of gax-java:
[INFO] Skipping execution
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Google Compute Engine Parent 1.15.1-SNAPSHOT:
[INFO]
[INFO] Google Compute Engine Parent ....................... SUCCESS [  0.845 s]
[INFO] proto-google-cloud-compute-v1 ...................... SUCCESS [  0.723 s]
[INFO] Google Compute Engine .............................. SUCCESS [06:45 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  06:49 min (Wall Clock)
[INFO] Finished at: 2022-11-15T10:41:49-05:00
[INFO] ------------------------------------------------------------------------
Finished Integration Tests for:
java-compute

It passes with the httpjson changes!

I'll need to complete part 2:
Search for returns streamacross googleapis repository to find all server-streaming call candidates (Need to make sure that the methods also don't havestream in their input parameter as that would be a bidi-stream method)
TODO: Search for a server-streaming call candidate

Added a few more tests cases. The tests cover:

Successful Unary Call (200 status code + Message returned)
Error Response from Unary Call (404 status code)
Cases for multiple responses from Unary Call
Null Content Success Response (200 status code + Null returned)
Null Content Error Response (400 status code + Null returned)
Non 4xx Error response

Since we're migrating to the gapic monorepo, I will create a patch file and apply the changes in the monorepo. Once that's been done I will close this issue. I'll reference this PR in that repo.


Implement NPC to change build

This would ease testing and would be fun

select class
select build

should change job
should allocate status point
should give stuff

close in favor of #55


Updated QD

It depends on this PR in RCL https://github.com/ros2/rcl/pull/858
@chapulina
Signed-off-by: ahcorde<EMAIL_ADDRESS>

@dabonnie , can we get this in? Thanks!


BME280 burst data readout

Hi,
I want to use IN100 to read data from BME280 sensor, but my current application expects to receive the data from all 8 registers instead of 6 as it is in the example. I tried to use burst read as recommended by the datasheet in section 4 but was unable to get it to work.
My current I2C configuration is:
"i2c": {
        "sw0PwrCtl": 1,
        "coldBootEn": 1,
        "warmBootEn": 1,
        "sclPin": 4,
        "sdaPin": 5,
        "addrBit": 7,
        "speed": 400,
        "slvAddr": 119,
        "memStoreLen": 8,
        "memStoreOffset": 0,
        "cmd": [
            "i2c wait: 3 1 27",
            "i2c tx: 3 f2",
            "i2c tx: 3 1",
            "i2c tx: 3 f5",
            "i2c tx: 3 0",
            "i2c tx: 3 f4",
            "i2c tx: 3 25",
            "i2c null:",
            "i2c wait: 3 1 4e",
            "i2c tx: 3 f7",
            "i2c null:",
            "i2c rx: 3",
            "i2c rx: 3",
            "i2c rx: 3",
            "i2c rx: 3",
            "i2c rx: 3",
            "i2c rx: 3",
            "i2c rx: 3",
            "i2c rx: 3",
            "i2c null:"
        ]
    },

Using the same command sequence on Raspberry Pi Pico works fine.
Would it be possible to add an example configuration for burst reading BME280 data registers?

Hi There,
Can you share more details about it not working? Are you seeing no data being read back at all? There is a known issue with multiple read commands being sent a row where at most 5 read commands will work before a new write command must be issued.

Yes, that was what was happening. IIRC when I looked at the signals on an oscilloscope, after the last write command (0xf7) there was no activity on SCL or SDA lines. Later today I can add a picture from the scope to show exactly what's going on.

As promised I'm attaching pictures from my scope.
Channel 1 - SCL
Channel 2 - SDA
Channel 3 - SW0
First, the whole communication sequence during the SW0 active period:

Zooming to the last command shows that after it's sent the SCL and SDA lines are idle and IN100 doesn't attempt to read the data at all:

I have also checked if the register configuration commands and data read commands are sent correctly. I can attach those pictures as well if needed.

If I understand correctly splitting the read into multiple shorter ones (e.g. 3 + 5 bytes) should fix this problem, am I right?

If I understand correctly splitting the read into multiple shorter ones (e.g. 3 + 5 bytes) should fix this problem, am I right?

Have u read the data successfully?


Labs Documentation Readability/Clarity

Purpose of PR
Adding readability/clarity to the labs.
Type of PR

[x] Documentation changes
[ ] Code changes
[ ] Test changes
[ ] CI-CD changes
[ ] GitHub Template changes
[ ] Codespaces changes

I've marked some TODOs that need clarifying before PR gets officially out of draft.


Tiling mode

Can you make a patch for tiling mode please?

Hello! I've been working on a pseudo-tiling function for a little bit now, and I could start a PR for a patch if interisted. Please note I am a novice when it comes to C programming, so it might not be great, and if so, please point out my errors.
PR is incoming if I can figure out how to do it.
Hope this helps.

I did a pull request for window splitting: #76 . Hope this helps.

Please close this issue, has been implimented by #76.


The generation of the sub-task 【fine-grained action】 in MVBench

Hello authors,
In your paper, you mention that the candidates of the question in the sub-task【fine-grained action】 are generated using UMT-L. Could you please clarify whether you use a pre-trained UMT-L model to encode the videos and the 339 categories (the total number of categories in Moments in Time dataset), and then compute the text-visual similarity?
Thank you!

Yes, we use the UMT-L model to encode the video, and then select the top-10 similar types based on the prediction score.


AFHTTPRequestOperation fails to return error when there is no connection

I'm attempting to use a simple AFHTTPRequestOperation to upload a file as the example shows in the documentation. This fails to produce an error when there is no connection. This looks to be because "- (NSError *)error" returns [super error], which returns _error because that's what the AFURLConnectionOperation returns. "_error" never gets set to the connection error, because:

(void)connection:(NSURLConnection *)__unused connection
didFailWithError:(NSError *)error

sets the error with "self.error = error" which sets it to _HTTPError. A quick fix is to have "- (NSError *)error" return _HTTPError instead of [super error].

You're absolutely correct. That [super error] was a vestige of a previous way that I was handling things. This is now fixed in 1de3d17


Create TMPower.cs as an abstract class which MagicPower and MightPower inherit from

This allows us to reuse code in other sections and reduces code overall.
I also went ahead and cleaned TMPower while I was at it. I also changed the project to target C#10 so that we can start getting rid of a lot of the overhead from linq functions that only use constants in their functions aka FirstOrDefault((Magic Power x) => x.abilityDef == 'ab') => FirstOrDefault(static mp => mp.abilityDef == 'ab') The reasoning behind this is compiler optimization where static makes it so the function is no longer anonymous to the program, but to us it is. Anonymous functions are slower than named ones.

No idea why this one decided to merge my branch into this branch, so I opened a new one that is fixed


ref(pkg/helm): package helm refactor

includes:

(#825) comptability interface between
old/pkg/helm and new/pkg/helm

(#827) helm client scaffolding and
rpc / release options

Initial commit for pkg/helm refactor. Currently in pkg/helmx but will be dropped in to pkg/helm once complete. See #807 for complete scope and task list.

Looks good!


type level virtual machine, enable profiling on toplevel code

This PR will try to setup a virtual machine, which simulates Julia code execution but only works on type level, and finally enable profiling on toplevel code.

okay, it turns out implementing JuliaInterpreter.jl-like interpretation on type-level is so hard. Hacking on surface AST (like hoisting and directly evaluating toplevel expressions and then wrap the other expression into a virtual lambda function) would be much more easier and good to be a first step


Add support for Xcode 7.1 beta

The plist change is good, but could you remove the binary file? I'm not sure what it is for.

Ok. Sorry. I'm unfamiliar with github.


Separate User entity and UserDetails implementation

UserDetailsEntity issue
Right now we have UserDetailsEntity, that implements UserDetails and also being Entity.
@Entity
@Table(name = "user_details_entity")
// ...
public class UserDetailsEntity implements UserDetails {
    // implementation
}

UserDetailsServiceImpl issue
And we have UserDetailsServiceImpl, that is solving two problems: service for users and service for UserDetails
It also need to be separated.
How to implement
The plan is something like this:

UserDetailsImpl is separate class, that is wrapping User from the database and using it's fields in own methods
UserDetailsService operates with UserService and maps User to UserDetailsImpl in the way that is described above

Done

[X] User and UserDetailsImpl are separate
[X] UserService and UserDetailsServiceImpl are separate


Add support for Beast WebSockets

I ran into https://github.com/laudrup/boost-wintls/issues/22, and having to add the termination code isn't ideal, so I added the code here. Since I'm not sure if all Boost installations (that have at least ASIO) will have beast, I added a check if beast is available and didn't include it in the beast/wintls.hpp wrapper (is that assumption true?).
I added tests for sync and async WebSocket servers to the echo test-case.

I have no clue how to fix the MinGW tests. I guess I'll need to split them up?

Thanks a lot for this.
Overall it looks really good.
If I understand this correctly, this implements an overload for boost::beast::teardown required to use this library with beast websockets?
I would definitely like to merge this, but it might make more sense to wait with this before #80 has been completed. There will be conflicts for sure no matter what get merged first though.
I have a few further comments, but please let me know if I've understood correctly that this PR makes use of beast websocket possible by implementing an overload or if I'm missing something.
Thanks.

If I understand this correctly, this implements an overload for boost::beast::teardown required to use this library with beast websockets?

Yes, that overload was missing for beast.

I would definitely like to merge this, but it might make more sense to wait with this before https://github.com/laudrup/boost-wintls/pull/80 has been completed. There will be conflicts for sure no matter what get merged first though.

That's totally fine, I'll watch the PR.

I think you might want to add
concurrency:
  group: unittest-${{ github.ref }}
  cancel-in-progress: true

to the unittest.yml action to stop in-progress actions when updating a PR.
Update: I added it (I can remove it before a merge).

I'm not sure what it does t be honest, but if it cancels builds that are outdated because of new commits then I guess it makes sense to keep it?

I'm not sure what it does t be honest, but if it cancels builds that are outdated because of new commits then I guess it makes sense to keep it?

This ensures that there can only be one active build on a given branch/tag/PR (from github.ref). We use it in another repo where the builds can take some time.

It would be very cool if you could add an example for how to use this library with websockets. I guess you could more or less copy one of the smallest examples for boost.beast and modify it for use with wintls?

I added two examples (from the sync and async examples from beast). I modified the CMakeLists.txt in the examples to make it easier to add examples.

@Nerixyz
It looks really good. Thanks a lot for your effort here. Two things:
Please add the websocket example to doc/examples.rst as well as files for the actual examples (eg. websocket_client.rst). Should hopefully be relative simple to copy and modify the existing examples. You can build the documentation by building the doc target on a non-windows system. If you cannot do that easily I'll have a look before merging.
Also, please split up your PR to separate commits. Something like the change to the Github actions workflows is not really related to the addition of websocket support. The same goes for the change to finding the root for OpenSSL when using MinGW (I have fixed the same in the master branch but your change looks cleaner). So at least three separate commits.
Thanks a lot once again.

You can build the documentation by building the doc target on a non-windows system. If you cannot do that easily I'll have a look before merging.

I built it on Windows and it looked fine. Is there a difference when building on non-Windows systems?

So at least three separate commits.

I split it up in a few more commits, but I can squash them if desired.

You can build the documentation by building the doc target on a non-windows system. If you cannot do that easily I'll have a look before merging.

I built it on Windows and it looked fine. Is there a difference when building on non-Windows systems?

I've only tried to build it on non-Windows systems since it's so much simpler for me. I rarely use Windows for anything :-)
It's very nice to know that the documentation can actually be built on Windows so thanks for that.

So at least three separate commits.

I split it up in a few more commits, but I can squash them if desired.

You might not have to split it into so many commits, but I definitely prefer too many commits than too few (for bisecting, cherry-picking etc.). Thanks a lot.
Looks great. Thanks a lot for all your work on this.


Inactive Users repo: Add unit tests

Background
The repo python code within the repo could do with a refactor and apply unit tests.
This is the current process that needs to be improved / automated better:
Run the Dormant GH Users scripts on the MoJ and AP GH Orgs, get the dormant export file from Tony, add it to the repo, run the script with false, email all the users that are not active, after a week or two, get another Dormant export file from Tony, run the script with false to check for any new users, add their usernames to the allow list temporarily, run the script with true, to remove the users that haven't logged into GH, expect users to email after to be added back into the GH Orgs.

Unit Test for for github_service.py
Unit Test for for slack_service.py


Update Contact Page

Contact Page is showing random text(Lorem Ipsum) rather than relevant one!!
Also Layout is not looking Good.
Assign this to me!!

@Kritika30032002 go ahead also make the theme better and fix night mode for it to remain same as current page

@Kritika30032002 go ahead also make the theme better and fix night mode for it to remain same as current page

okay @ElvinPero

Dark mode is not added
You can assign me that.

@Kritika30032002 Can you work on both isssues at same time

Or would u like to pass this one

@Kritika30032002 Can you work on both isssues at same time

Yes!!

@Kritika30032002 Are you working on this?

@ElvinPero  yes I am working on it.
You can assign only enhancement of dark mode to devjain if you want

@Kritika30032002 Hey whats the progress on this one

its done along with dark mode enabled

@Kritika30032002 Hey whats the progress on this one

@ElvinPero   PR has been sent  #100
Dark mode also enabled!!

@Kritika30032002 Is your PR ready?

Please assign to me I have already made a pull request you can check it for enabling dark mode.

@harshilshah99 @Kritika30032002 Had said in morning she will send a PR which hasnt come yet,
If her PR doesnt come by tomorrow morning I will assign you and some changes have to be made in your commit which I will tell you.

@harshilshah99 @Kritika30032002 Had said in morning she will send a PR which hasnt come yet, If her PR doesnt come by tomorrow morning I will assign you and some changes have to be made in your commit which I will tell you.

Ok, I will surely make changes if you assign it to me.


Logo

Hi. I am a graphic designer. I would like to volunteer to design a logo for this repository that you can use in the readme file. What do you say?

Hello @batarian71 ,
PR are always welcome.
Regards

I designed an atom model that is different from the general atom models. What do you think?


Can't run SGAN and adaptation.

Seems like there are multiple bugs in the SGAN code.
I tried to fix them but I am not familiar with Keras yet so i can't fixed them all by my self.
Thanks in advance for any advice or update. :)
PS. Also, I would like to adapt to model to use with other data (person-based data, a row means a person with about 200 features/cols and output labeled as 0 or 1). It would be nice if you please clarify how to do that as well.

What bugs are you referring to?


Laravel-echo-server stops working without an error in logs

I've been working for almost 1 year with laravel-echo-server for my laravel project. The project was working perfectly and it was being used for real-time communication between the clients and the server to show real-time server usage and some kind of chats.
I've created a .service file in the server  using CentOS where the project is located. After a while working in production, the laravel-echo-server stopped working until I restarted the service. I thought that it could be like a "random problem" in my server, and I ignored it.
This year, the service stopped working 2 times and still no errors in logs (take a look at the screenshot attached below). If you take a look at the picture where the last error ocurred, you can see that the service stopped working on September, 2nd until I restarted it on September, 11th, and these are the logs reported by journalctl.
Now I'm facing that I have to do something with this, because even if it only occurs every 6-8 months, the whole system stops working and it is not something that we can assume in production. I do not know if anyone experienced the same problem or if you are aware of this and have a solution for it. I've tried to Google this problem, but I couldn't find anything related.
This is the server structure:

Centos 7
Apache & PHP 7
Nodejs 6.17.1
Laravel 5.6
Laravel-echo-server 1.2.8

I do not know if there is a fix for this in a further versions, I'll be waiting for your answers.
Thanks for your time.

you can use supervisor to keep the echo-server running.

i use supervisor and a cronjob for a daily restart
0 4 * * * supervisorctl restart laravel-echo

and check your failed_jobs log

Thanks for the information.
This question may sound stu***, but how should a Laravel failed job have some influence on node socket server like laravel-echo-server? I mean that jobs deliver something through sockets or not (if doesnt pass a check or if an error is thrown).

I can remember i had some problems with huge objects and errors inside the event classes crashed my echo server, also the server is in version 1.5 maybe updating fix your problems.

Answering to @shafter suggestions of checking errors in jobs, etc. I did not have any errors in my jobs. In fact, there were no failed jobs since February this year and it was because the email service from the domain provider stopped working.
Also, I did not send huge objects. Only useful and required informations (3 to 6 object properties). As you say. I will try to update it in my dev environment and, if nothing breaks up, I'll update on production, but if there is an error or something that stops the server working, even if it is up, should be notified. For example, if they setup a timeout config that we can modify and if a request takes more than that timeout, throw an exception and we can get some information about when happened, the event name or something like that and keep working for other events instead of getting freezed.
Is just a suggestion...


Keyboard support or event with "istrusted"

Hi
would it be possible to add support to keyboard events or the possibility to trigger events that are "trusted", as keyboards events 'or other kinds of events) triggered by javascript can be detected as "not trusted" and not accepted by the website

+1

actually i have found how to do that :
$page->getSession()->sendMessageSync(new HeadlessChromium\Communication\Message('Input.dispatchKeyEvent', [
//'type'	=>	'event.keyDown',
'type'	=>	'char',
'text'	=>	'a'
]));
for each letter to press


Dropdown (multiple) search box does not grow with input

Currently when you have a multi select dropdown, the search box does not expand when you start typing in it.

In Semantic UI, the width style property is set on the DOM element itself when the content of the search box changes. We could take a similar approach here?

We had pushed this minor issue off previously.  Time to fix it! See #371.

Fixed and released in v0.27.7.


Migration to aioredis 2.0.0

Hi, are there any plans to migrate to aioredis 2.0.0?
I am running into occasional ConnectionResetError and this issue suggests 2.0.0 might fix it (maybe...)

+1
@mariaines How did you end up proceeding? redis-py-cluster looks like a potential alternative but its not asyncio.

Are you sure the problem is with aioredis and not with the number of connections on redis node? Check connected_clients and rejected_connections in INFO command output. Also check maxclients config option if connected_clients near on this limit. Also make sure you have enough network resources (number of ports, limits, TCP_NOWAIT states, etc.) on the client host and redis itself.
ConnectionResetError usually means network level problems. this is in fact receive of the RST packet.

And answer the question. There are no plans to migrate to aioredis>=2.


What's the difference between this project and occ

I am watching this project. It's very good that this's implemented in rust. And I wonder what's the major difference between this and opencascade, is there any better design inside this project?

The most significant difference lies in the small-scale crating. Overall, it is a competitor to OCCT, but the truck-platform by itself, for example, is aimed at replacing three.js in the WebGPU generation, or a primitive Unity that uses crates.io as an asset store. This design allows experts to replace individual implementations.. Moreover, it can easily accommodate to algorithm alternatives such as GPU accelerators for NURBS meshing, or unpredictable expansion for unknown new technologies. This is possible because of centralized package management by Cargo.

The most significant difference lies in the small-scale crating. Overall, it is a competitor to OCCT, but the truck-platform by itself, for example, is aimed at replacing three.js in the WebGPU generation, or a primitive Unity that uses crates.io as an asset store. This design allows experts to replace individual implementations.. Moreover, it can easily accommodate to algorithm alternatives such as GPU accelerators for NURBS meshing, or unpredictable expansion for unknown new technologies. This is possible because of centralized package management by Cargo.

Is there any plan to support boolean operations? Glad to implement these.

Is there any plan to support boolean operations? Glad to implement these.

Yeah, we also consider that boolean operations are necessary and is the next plan after STEP handling.
We cannot make any official promises about deadlines, however, I'm hoping to release something by August.

Yeah, we also consider that boolean operations are necessary and is the next plan after STEP handling.
We cannot make any official promises about deadlines, however, I'm hoping to release something by August.

Another very important difference is that this project is Apache 2.0 licensed, while OCC is LGPL 2.1.

Close this issue. Boolean operations will be discussed in #17 .


Is the 'count' method executed in the database or in RAM?

unread = await dexie_db.email
.where("folder_name")
.equals("INBOX")
.filter(email => email.flags.indexOf("SEEN") == -1 )
.count();
I hope to retrieve the number of unread emails in the inbox using the above method. However, I'm concerned that it might impact browser performance when dealing with a large volume of emails.

when only using a 'where' clause count is executed natively in IndexedDB. In your example there is a filter attached and this makes dexie have to count the rows manually instead. It's doing this by iterating every row that matches the where clause and testing it against the provided JS filter.
If there are thousands of unread emails in user's inbox, the count could take some time (500 ms or so).
To optimize this you could flag unread messages differently. Right now you are using 'flags'. Even if it could be indexed using multiEntry index, it cannot be a part of a compound index. So if you could complement this flag with an indexed property such as 'last_opened' (a timestamp), you could have a compound index of [folder_name+last_opened] and query the count as such:
// count all unseen emails
unread = await dexie_db.email
.where({
  folder_name: "INBOX",
  last_opened: 0 // make sure upgrade db and set this property on all items.
})
.count();

// count all seen emails:
unread = await dexie_db.email
.where("[folder_name+last_opened]")
.between(["INBOX", 1], ["INBOX", Dexie.maxKey])
.count();

when only using a 'where' clause count is executed natively in IndexedDB. In your example there is a filter attached and this makes dexie have to count the rows manually instead. It's doing this by iterating every row that matches the where clause and testing it against the provided JS filter.
If there are thousands of unread emails in user's inbox, the count could take some time (500 ms or so).
To optimize this you could flag unread messages differently. Right now you are using 'flags'. Even if it could be indexed using multiEntry index, it cannot be a part of a compound index. So if you could complement this flag with an indexed property such as 'last_opened' (a timestamp), you could have a compound index of [folder_name+last_opened] and query the count as such:
// count all unseen emails
unread = await dexie_db.email
.where({
  folder_name: "INBOX",
  last_opened: 0 // make sure upgrade db and set this property on all items.
})
.count();

// count all seen emails:
unread = await dexie_db.email
.where("[folder_name+last_opened]")
.between(["INBOX", 1], ["INBOX", Dexie.maxKey])
.count();

You could also use a property like "seen" and set it to 1 or 0. The reason I picked a timestamp was just in case this info could be relevant in other scenarios such as listing the 10 most recently opened emails with a natively index based query (where-clause based with a .limit(10)).

In the example above, flags are also an index, but I'm not sure if there's a possibility to optimize the search for folder_name while also searching if flags contain 'SEEN'.

No, as I explained, it's not possible to combine a multiEntry index (flags) with a normal index (folder_name) for the purpose of counting, but it is with a compound index containing two single-valued properties.

No, as I explained, it's not possible to combine a multiEntry index (flags) with a normal index (folder_name) for the purpose of counting, but it is with a compound index containing two single-valued properties.

Does this mean that later on, if I want to perform fuzzy searches on the content serving as the index, I can only retrieve it through filters in RAM?

I don't understand the question. You can still use the index for flags as you've been doing before.

I don't understand the question. You can still use the index for flags as you've been doing before.

Assuming I have an index named 'content',And 'content' is an index containing a piece of text.Now, I want to perform a fuzzy search using a word or a sentence,In this scenario, does it necessitate pulling all email data into memory for computation?

I don't understand the question. You can still use the index for flags as you've been doing before.

Sorry, my English is not very good.The description of the issue might be somewhat inaccurate.

I don't understand the question. You can still use the index for flags as you've been doing before.

Assuming I have an index named 'content',And 'content' is an index containing a piece of text.Now, I want to perform a fuzzy search using a word or a sentence,In this scenario, does it necessitate pulling all email data into memory for computation?

If doing .filter() you can do fuzzy search without loading all records into RAM but loading them one-by-one and testing against your filter. This will not load all records to RAM but it is not as fast as loading several records into RAM and filtering afterwards - but again - that takes more memory.
To optimize both RAM and memory load the table in chunks of 100 or so (depending on how much RAM you can allow and how big each object is) and search in the array chunks. To do that you need to loop through the chunks as follows:
const CHUNKSIZE = 100;
const result = [];
let startKey = -Infinity;
while (true) {
  const chunk = await db.yourTable.limit(CHUNKSIZE).toArray();
  result.push(...chunk.filter(yourFilteringFunction));
  if (chunk.length < CHUNKSIZE) break; // done
  startKey = chunk.at(-1).id; // Replace .id with your primary key property.
}

This way you never load all records to memory but still do a pretty fast fuzzy search.

const CHUNKSIZE = 100; // Could be 1000 also depending on how much RAM you are willing to sacrifice.
const result = [];
let startKey = -Infinity;
while (true) {
  const chunk = await db.yourTable
    .where(':id').above(startKey)
    .limit(CHUNKSIZE)
    .toArray();
  result.push(...chunk.filter(yourFilteringFunction));
  if (chunk.length < CHUNKSIZE) break; // done
  startKey = chunk.at(-1).id; // Replace .id with your primary key property.
}

I don't understand the question. You can still use the index for flags as you've been doing before.

Assuming I have an index named 'content',And 'content' is an index containing a piece of text.Now, I want to perform a fuzzy search using a word or a sentence,In this scenario, does it necessitate pulling all email data into memory for computation?

If doing .filter() you can do fuzzy search without loading all records into RAM but loading them one-by-one and testing against your filter. This will not load all records to RAM but it is not as fast as loading several records into RAM and filtering afterwards - but again - that takes more memory.
To optimize both RAM and memory load the table in chunks of 100 or so (depending on how much RAM you can allow and how big each object is) and search in the array chunks. To do that you need to loop through the chunks as follows:
const CHUNKSIZE = 100; // Could be 1000 also depending on how much RAM you are willing to sacrifice.
const result = [];
let startKey = -Infinity;
while (true) {
  const chunk = await db.yourTable
    .where(':id').above(startKey)
    .limit(CHUNKSIZE)
    .toArray();
  result.push(...chunk.filter(yourFilteringFunction));
  if (chunk.length < CHUNKSIZE) break; // done
  startKey = chunk.at(-1).id; // Replace .id with your primary key property.
}

This way you never keep more than CHUNKZISE records in memory at the same time but still do a pretty fast fuzzy search.

Thank you for your help


feat(dev): don't clean up assets when in dev-server

SUMMARY
Cleanup/override static assets only when running npm run build or npm run dev, not when in npm run dev-server. This allows developers to compare the current version of Superset with what they are working on without switching back and forth between branches.
A typical workflow would be:
First, run production build and start the Flask server:
npm run build
superset run -p 8088

Then start the dev server in another window:
npm run dev-server

Now you can visit http://localhost:8088 for the production version, and http://localhost:9000 for the dev version. Any local changes you made wouldn't affect the production version, meaning you will always have a production version to refer to when you are actively developing something.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
N/A
TEST PLAN
Manual. Follow the workflow above and make sure both debug servers work.
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #11648 (9b89144) into master (777a84c) will decrease coverage by 8.23%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master   #11648      +/-   ##
==========================================
- Coverage   62.26%   54.03%   -8.24%
==========================================
  Files         873      406     -467
  Lines       42281    14320   -27961
  Branches     3964     3683     -281
==========================================
- Hits        26327     7738   -18589
+ Misses      15774     6406    -9368
+ Partials      180      176       -4

Flag
Coverage Δ

cypress
54.03% <ø> (?)

javascript
?

python
?

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

...uperset-frontend/src/dashboard/util/dnd-reorder.js
0.00% <0.00%> (-100.00%)
:arrow_down:

...rset-frontend/src/dashboard/util/getEmptyLayout.js
0.00% <0.00%> (-100.00%)
:arrow_down:

...dashboard/components/resizable/ResizableHandle.jsx
0.00% <0.00%> (-100.00%)
:arrow_down:

.../src/dashboard/util/getFilterScopeFromNodesTree.js
0.00% <0.00%> (-93.48%)
:arrow_down:

...src/dashboard/components/gridComponents/Header.jsx
10.52% <0.00%> (-86.85%)
:arrow_down:

...rc/dashboard/components/gridComponents/Divider.jsx
13.33% <0.00%> (-86.67%)
:arrow_down:

...ontend/src/dashboard/util/getDashboardFilterKey.ts
14.28% <0.00%> (-85.72%)
:arrow_down:

...set-frontend/src/views/CRUD/welcome/EmptyState.tsx
5.71% <0.00%> (-82.10%)
:arrow_down:

...et-frontend/src/components/ListView/ActionsBar.tsx
11.11% <0.00%> (-81.20%)
:arrow_down:

...et-frontend/src/components/Menu/LanguagePicker.tsx
0.00% <0.00%> (-80.00%)
:arrow_down:

... and 775 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 777a84c...9b89144. Read the comment docs.


Change Enter Full Screen to Exit Full Screen once in full screen mode

Just a tiny thing.
By the way, the updated version of director that comes with drake now is amazing!

I think this might be a feature of your window manager, not a feature provided by the app?  Can you show a screenshot?

I don't care enough about this to investigate further :-) Sorry to bother you.


How to quit UI?

I'm sure this is a dumb question, but how does one close/escape the UI once they are in it? Thank you!

Ctrl-C


High rate of packet loss and latency in a UDP multicast scenario

What happened?

High rate of packet loss and latency in a UDP Multicast Scenario with low bitrate.
How to reproduce it?

Two kubernetes nodes with weave net
One POD (ffmpeg) produces 4 multicasts (same IP and 4 different ports; e.g <IP_ADDRESS>:2000, <IP_ADDRESS>:2001, <IP_ADDRESS>:2002, <IP_ADDRESS>:2003) output with an overall of around 10 Mbps. Let it be the "producer"
n PODs (shaka) consuming the same 4 multicasts. Let them be the "consumers"

Two scenarios:

Producer runs in node 1 and consumers run in node 2. With 1 consumer, everything is ok. But if I move to 10 consumers, all the veth interfaces drop quite a few packets.
Producer and consumer run in the same node. There are not any dropped packets, but there is a high latency.

The MTU is set by default to 1376 as the underlying machine is a VM with adapters MTU = 1500
Is there a BW limitation to prevent more than one consumer to listen to multicast? I have verified that all "n" consumers are doing JOIN succesfully. Should I configure anything
Versions:

$ weave version
2.5.1
$ docker version
18.06.0-ce
$ uname -a
Linux k8s-master3 4.4.0-142-generic #168-Ubuntu SMP Wed Jan 16 21:00:45 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
$ kubectl version
1.13.2

Logs:
$ kubectl logs -n kube-system <weave-net-pod> weave

Nothing relevant in logs

Producer runs in node 1 and consumers run in node 2. With 1 consumer, everything is ok. But if I move to 10 consumers, all the veth interfaces drop quite a few packets.

@masantiago  Weave for multicast uses broadcast underneath. So each node would receive the multicast packets irrespective of any pod joining the multicast group running on the node or not. So ideally it should not matter its 1 or n consumer. Are these 10 consumers running in same node? Do you see packet drops even if they are spread across the cluster. When do you see most packet drops? any other pattern/information will be useful.

Thank you  for your answer@murali-reddy.
Find below two scenarios:

1 producer in node A. 1 consumer in node B. The veth that is sending the traffic to the POD is:

vethwepl4a8b280 Link encap:Ethernet  HWaddr 9e:f9:37:d7:4a:32
inet6 addr: fe80::9cf9:37ff:fed7:4a32/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1376  Metric:1
RX packets:3 errors:0 dropped:0 overruns:0 frame:0
TX packets:506406 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:150 (150.0 B)  TX bytes:417741116 (417.7 MB)
The statistics for that veth is: (using nload)
Avg: 8.93 Mbit/s
Min: 6.11 Mbit/s
Max: 11.69 Mbit/s

1 producer in node A. 10 consumers in node B. The same veth turns to in a few seconds in:

vethwepl4a8b280 Link encap:Ethernet  HWaddr 9e:f9:37:d7:4a:32
inet6 addr: fe80::9cf9:37ff:fed7:4a32/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1376  Metric:1
RX packets:3 errors:0 dropped:0 overruns:0 frame:0
TX packets:917815 errors:0 dropped:609 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:150 (150.0 B)  TX bytes:754565754 (754.5 MB)
The statistics remain more or less the same: (considering that the video encoding evolves)
Avg: 8.27 Mbit/s
Min: 3.45 Mbits
Max: 11.78 Mbit/s
If I have a look to the rest of veth of the other PODs, they all have dropped packets.
Therefore, this is the case for consumer and producers in separate nodes. What is weirdest, is the case when I co-locate consumer and producers in the same node. There are not any drops but the latency of UDP packets increases and my video service cannot work. I can go to more detail on this scenario in another post.
To put all the context, I am testing this with two Kubernetes nodes that are actually 2 x VMs of VirtualBox with Bridge adapter. Also, I increased the Linux buffers just in case to up to 50MB:
sudo sysctl net.core.rmem_max=52428800
sudo sysctl net.core.rmem_default=52428800
sudo sysctl net.core.wmem_default=52428800
sudo sysctl net.core.wmem_max=52428800
I do not know where it is the bottleneck: weave, VM, other,... Hope you can help me out.

Thanks for the details.

I do not know where it is the bottleneck: weave, VM, other,... Hope you can help me out.

Weave's implementation of multicast is not optimal at the moment (see https://github.com/weaveworks/weave/issues/178). There is unnecessary overhead on the nodes when there is no receivers. That's the only issue. This does not apply to your setup.
Weave configures OVS (to broadcasts across the nodes) and L2 multicast ethernet address to broadcast across the pods connected to weave bridge. This should scale easily to the data rates you are sending given there in enough processing power.
Not sure the configuration of virtual-box VM's, but i would rule out any issues due to virtual box by running on bare-metal and see how it compares.

I do agree, testing on bare-metal would be a key.
Honestly, what drove me to think of any problem of weave was the third scenario I put forward at the very beginning:

1 producer in node B. 10 consumers in node B. Both are co-located. In this case, there are neither dropped packets nor overrun. But the result is that the consumers are receiving the UDP packets with a high delay. Those consumers are packaging video and it is clear how the outcome is delivered very late.

Does it make sense? How can it have anything to do with VM if it is inside the machine? Do you find any implication in weave thinking that 10 PODs are receving the same multicast in every veth adapter?
Take for granted that machine is healthy in both CPU and RAM.
Thank you very much for all your help.

1 producer in node B. 10 consumers in node B. Both are co-located. In this case, there are neither dropped packets nor overrun. But the result is that the consumers are receiving the UDP packets with a high delay. Those consumers are packaging video and it is clear how the outcome is delivered very late.

What is the latency you are observing?
In the co-located scenario, all the pods are connected to same weave bridge, Weave does not even configure anything on the bridge for multicast. Only for packets that are to be sent out to other nodes weave configures OVS datapath. But in this case its just Linux bridge handling L2 multicast ethernet packets. Its a soft-switch so there will be overhead (depending on pps) but should not be high for the data rates you are sending.

Does it make sense? How can it have anything to do with VM if it is inside the machine?

May be not. But we need a reference to compare against.

Do you find any implication in weave thinking that 10 PODs are receving the same multicast in every veth adapter?

No. At least in this case it just Linux bridge involved.
Does making producer send lower pps has any effect interms of latency in this case and dropped packets are inter-node case?

Hi again. After some analysis, I found out that the co-located scenario was actually failing because of the bottleneck of the hard disk.
Therefore, all my effort is now addressed to the scenario of different nodes: 1 producer in node A. 10 consumers in node B. As said, with around 100 Mbps (10 x 10 Mbps/channel) all the veth start dropping packets.
I have changed to use host network in the PODs, both in the producer and the consumer. There is not any dropped package in this case. That demonstrates that it has something to do with the weave overlay network. Do you suggest any observation in any logs?
I have even taken care of the MTU size. It is now configured to MTU = 1410 in the veth, to give room to the headers of the overlay. And, more over, the UDP packets are sent of 1344 bytes (7x188 payload + 8 UDP header + 20 IP header).

I have changed to use host network in the PODs, both in the producer and the consumer. There is not any dropped package in this case. That demonstrates that it has something to do with the weave overlay network. Do you suggest any observation in any logs?

Do you see any dropped packets on vethwe-datapath or vethwe-bridge?  Weave ensures a copy of multicast packet is sent over vethwe-datapath <->vethwe-bridge pair connecting weave bridge. Afterwards its linux bridge that does the job of sending the packets on veth interfaces to the pods.
Just to see if bridge being bottleneck, can we spread to consumers to two different nodes?

Do you see any dropped packets on vethwe-datapath or vethwe-bridge? Weave ensures a copy of multicast packet is sent over vethwe-datapath <->vethwe-bridge pair connecting weave bridge. Afterwards its linux bridge that does the job of sending the packets on veth interfaces to the pods.

Yes, both. One in RX and TX in the other. Do not pay too much attention to the different number, because I may have rebooted or update the MTU. Find below the data for the node where consumers are located.
vethwe-bridge Link encap:Ethernet  HWaddr 4a:14:8a:06:42:43
inet6 addr: fe80::4814:8aff:fe06:4243/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1410  Metric:1
RX packets:73101219 errors:0 dropped:14298 overruns:0 frame:0
TX packets:370469 errors:0 dropped:11 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:94164964198 (94.1 GB)  TX bytes:1327008051 (1.3 GB)
vethwe-datapath Link encap:Ethernet  HWaddr fe:84:d1:e1:6b:c1
inet6 addr: fe80::fc84:d1ff:fee1:6bc1/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1410  Metric:1
RX packets:370469 errors:0 dropped:22 overruns:0 frame:0
TX packets:73101219 errors:0 dropped:7149 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1327008051 (1.3 GB)  TX bytes:94164964198 (94.1 GB)
Nevertheless, the weave adapter is free of errors in the same node.
weave     Link encap:Ethernet  HWaddr da:77:d3:9b:d6:85
inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
inet6 addr: fe80::d877:d3ff:fe9b:d685/64 Scope:Link
UP BROADCAST RUNNING MULTICAST  MTU:1410  Metric:1
RX packets:73315637 errors:0 dropped:0 overruns:0 frame:0
TX packets:491727 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:93471805701 (93.4 GB)  TX bytes:1392778738 (1.3 GB)
Does it make any sense to you?

Hi, can you get the overall status out of one the Weave Net daemons as outlined at https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#troubleshooting ?
(There are two options there - one via a script and one via curl)
Then also do weave status connections or the equivalent curl ... /status/connections

Hi, can you get the overall status out of one the Weave Net daemons as outlined at https://www.weave.works/docs/net/latest/kubernetes/kube-addon/#troubleshooting ?

kubectl exec -n kube-system weave-net-g49kt -c weave -- /home/weave/weave --local status connections
-> <IP_ADDRESS>:6783     established fastdp 3a:29:40:a7:14:76(k8s-master2) mtu=1410
-> <IP_ADDRESS>:6783     established fastdp ba:fc:80:e9:b1:ef(k8s-master1) mtu=1410
-> <IP_ADDRESS>:6783     failed      cannot connect to ourself, retry: never


FEATURE: Configurable navigation button class attributes

Resolve: #20

Just marked @kdambekalns as a reviewer. When we merge this one, Neos.Form is once again down to 0 open PRs 🥇


Remove nomenclature.yaml

As mentioned in the final comment of #2, we should remove nomenclature.yaml from this repo.
My reasoning is that, for the future I would imagine that a lot of projects would use a nomenclature.yaml like so:
repositories:
  common-definitions:
    url: https://github.com/IAMconsortium/common-definitions.git/
  legacy-definitions:
    url: https://github.com/IAMconsortium/legacy-definitions.git/
definitions:
  variable:
    repository:
      - common-definitions
      - legacy-definitions

This would, as it stands fail since legacy-definitions currently contains common-definitions so the above setup would double common-definitions leading to artificial duplicate code entries.
Therefore my suggestion would be to remove nomenclature.yaml.
The downside to that is that we would loose our pseudo integration test which currently runs on this repo and imports common-definitions to see if they are compatible with legacy-definitions.
However, it might be better anyway to create a new repo in IAMconsortium just for this integration test purpose. This repo would contain no definitions of its own but rather just pull from both common as well as legacy definitions. I would then implement a trigger so that this runs on any change to either common or legacy definitions.
What are your thoughts @danielhuppmann?

Yes, agree to remove nomenclature.yaml.
But rather than creating yet another integration-test repository, wouldn't it be enough to have an integration-test workflow that "manually" adds an external repo (not via nomenclature.yaml but programmatically)? Either way, the fiddling to trigger the workflow on pushes to another repo remains.

True, I can just create the file on the fly as part of the test workflow, good idea.

Closing since implemented per https://github.com/IAMconsortium/legacy-definitions/commit/8179cac45a799b9e7ae9e3cd74ce240620eaeb44 and https://github.com/IAMconsortium/legacy-definitions/commit/e55780bfadbe2c412c71b1a7b5dc74c8f77464b9


fixed a few typos

Just fixed a few additional typos.

Hi @DavidHoerster, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;


[watchdisneychannel.go.com] Add support for locked episodes (with TV-Provider login)

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like that [x])
Use Preview tab to see how your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.01.05. If it's not read this FAQ entry and update. Issues with outdated version will be rejected.

[X] I've verified and I assure that I'm running youtube-dl 2017.01.05

Before submitting an issue make sure you have:

[X] At least skimmed through README and most notably FAQ and BUGS sections
[X] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[X ] Bug report (encountered problems with youtube-dl)
[X] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add -v flag to your command line you run youtube-dl with, copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):

C:\Users\rumag\Desktop\youtube-dl>youtube-dl.exe -f best --embed-subs --all-subs --convert-subs srt --username<EMAIL_ADDRESS>--password july1987 --ap-username<EMAIL_ADDRESS>--ap-password july1987 --ap-mso DTV http://watchdisneychannel.go.com/girl-meets-world/video/vdka3613338/03/19-world-meets-girl -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['-f', 'best', '--embed-subs', '--all-subs', '--convert-subs', 'srt', '--username', 'PRIVATE', '--password', 'PRIVATE', '--ap-username', 'PRIVATE', '--ap-password', 'PRIVATE', '--ap-mso', 'DTV', 'http://watchdisneychannel.go.com/girl-meets-world/video/vdka3613338/03/19-world-meets-girl', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp850, pref cp1252
[debug] youtube-dl version 2017.01.05
[debug] Python version 3.4.4 - Windows-10-10.0.14393
[debug] exe versions: ffmpeg N-82889-g54931fd
[debug] Proxy map: {}
[Go] 3613338: Downloading JSON metadata
[Go] 3613338: Downloading JSON metadata
ERROR: Unable to download JSON metadata: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)> (caused by URLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (ssl.c:600)'),))
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp023qj2d\build\youtube_dl\extractor\common.py", line 406, in request_webpage
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp023qj2d\build\youtube_dl\YoutubeDL.py", line 2001, in urlopen
File "C:\Python\Python34\lib\urllib\request.py", line 464, in open
File "C:\Python\Python34\lib\urllib\request.py", line 482, in _open
File "C:\Python\Python34\lib\urllib\request.py", line 442, in call_chain
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp023qj2d\build\youtube_dl\utils.py", line 1041, in https_open
File "C:\Python\Python34\lib\urllib\request.py", line 1185, in do_open
...
<end of log>

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: http://watchdisneychannel.go.com/girl-meets-world/video/vdka3613338/03/19-world-meets-girl'
Single video: http://watchdisneyxd.go.com/star-wars-rebels/video/vdka3615846/03/11-ghosts-of-geonosis-part-1
Playlist: http://watchdisneychannel.go.com/disney-mickey-mouse/video/vdka3572406/03/101-duck-the-halls-a-mickey-mouse-christmas-special#playlist=PL551978168

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

Description of your issue, suggested solution and other information
Explanation of your issue in arbitrary form goes here. Please make sure the description is worded well enough to be understood. Provide as much context and examples as possible.
If work on your issue requires account credentials please provide them or explain how one can obtain them.
PS: It works fine with episodes that doesn't require TV-Provider login, but it doesn't work when you have to.

https://github.com/rg3/youtube-dl/issues/11573#issuecomment-269896421


now_utc() not found

Hey guys, I think I've found a bug - OffsetDateTime::now_utc() not found.

Error details
error[E0599]: no function or associated item named `now_utc` found for struct `time::OffsetDateTime` in the current scope
   --> C:\Users\Martin\.cargo\registry\src\github.com-1ecc6299db9ec823\cookie-0.14.0\src\lib.rs:854:42
    |
854 |         self.set_expires(OffsetDateTime::now_utc() + twenty_years);
    |                                          ^^^^^^^ function or associated item not found in `time::OffsetDateTime`

error[E0599]: no function or associated item named `now_utc` found for struct `time::OffsetDateTime` in the current scope
   --> C:\Users\Martin\.cargo\registry\src\github.com-1ecc6299db9ec823\cookie-0.14.0\src\jar.rs:229:48
    |
229 |             cookie.set_expires(OffsetDateTime::now_utc() - Duration::days(365));
    |                                                ^^^^^^^ function or associated item not found in `time::OffsetDateTime`

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0599`.
error: could not compile `cookie`.

I think the problem is old time dependency:
https://github.com/SergioBenitez/cookie-rs/blob/9b165d6fda6f707c9799c5d1794f782691ce6e4c/Cargo.toml#L22
Version 0.2.6 doesn't have method OffsetDateTime::now_utc - see docs.rs. The method is available in time >= 0.2.11

cookie v. 0.13.3 works great, the problem is only in 0.14.0.
I've found the issue while I was upgrading dependencies in Seed. Tested on Windows; target folder and deps cache deleted.

Thank you!

A cargo update should get you a version of time with the method as 0.2.6 implies >= 0.2.6, < 0.3.0. Nevertheless, we should explicitly request a version of time with the methods to avoid any issues like this. Thanks for reporting.

0.14.1 is out with the proper time version string.


docs: update docs

:tada: This PR is included in version 1.1.0 :tada:
The release is available on npm package (@latest dist-tag)
Your semantic-release bot :package::rocket:


github.com/gorilla/websocket: package context, unrecognized import path "context" (import path does not begin with hostname)

Package upgrade signature, required context on new version
Ref
https://travis-ci.org/ReactiveX/RxGo/jobs/448246104

This issue is caused because we are using a very old Go version.
I suggest going to 1.9 released on 24 August 2017 or even more recent.


fix(example): add installation of python-etcd in Pytorch example

What this PR does / why we need it:
Add installation of python etcd library in the example Dockerfile of pytorch elastic training.
Which issue(s) this PR fixes (optional, in Fixes #<issue number>, #<issue number>, ... format, will close the issue(s) when PR gets merged):
Fixes #2050
Checklist:

[ ] Docs included if any changes are user facing

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 35.317%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
4379

Relevant Lines:
12399

💛  - Coveralls


Showing the player in ListView

Hi. First of all thank you for such a great library. I am using it in one of my project. I am showing the media player in ListView's row.
PROBLEM 1: I cannot use previous instance of AudioWife class. I solved it by returning new instance every time I call getInstance(). Am I doing correct? Is there any other way?
PROBLEM 2: Suppose audio is playing, now I scrolled the list down, now I scrolled it up. Now the audio is still playing and the seekbar is also moving which is good, but the time is not visible and also play and pause both icons are visible.
Please help me solve these two issues.

  AudioWife audioWife = new AudioWife();
        audioWife.init(mContext, Uri.parse("PATH")).
                useDefaultUi(holder.viewRec, ((Activity) mContext).getLayoutInflater());


Question on validity of SA1413

Hi,
Just curious how the StyleCopAnalyzers specific versus the original StyleCop rules are debated and assessed? Its possible I am missing the obvious but I wonder how SA1413 actually improves maintainability? I feel it makes code confusing leaving a developer to wonder if someone possibly removed a line in error? With the trailing comma removed it requires a more deliberate change.

All of these rules are implemented because of subjective reason. The reasons for implementing this rule can be seen in this issue: https://github.com/DotNetAnalyzers/StyleCopAnalyzers/issues/1121.

I did find that thread but it does not explain how the rule assists in maintainability. I can discount the weight of my opinion, I am only a grateful user who doesn't contribute but this rule is the first rule I can honestly say I didn't see merit in but actually thought it promoted the opposite. For even the stylistic rules that are subjective, I have always been able to justify them on the premise that consistency and uniformity is important.
The real purpose of my inquiry was simply to learn how the rules are voted in and discussed. My original post describes how I see this rule to be counter productive and promoting confusion.

@jcasale Note that this is a maintainability rule as opposed to a readability rule. The advantages of including the comma have to do with the work required to make a change to the item falling at the end of a list (removing it, adding to it, or reordering it). With SA1413 in place, these changes are slightly less work for the developer on average.
I voted in favor of the rule for the following combination of reasons:

I've hit both of the two cases originally described with the proposal multiple times
When I encountered the same situation but a trailing comma was in place, it was less work to make whatever change I was doing

We don't have a formal process defined/documented for new rules. StyleCop is opinionated on a subject matter that the compiler really doesn't care about, so typically we try to weigh the following to determine what/if opinion we'll have:

The likelihood of a situation applying to a user's code
Whether or not StyleCop Analyzers already has a rule covering the situation
Whether or not StyleCop Analyzers already has a rule covering a similar situation, such that the opinions on somewhat related rules can be consistent

@sharwell Thanks for the detailed reply.

Sam recommended that I turn off this rule if I don't like it.  I think your rational is backwards.  At this stage in the product's maturity, you should be telling the people who want this new rule to turn it on.  There is a significant well of resentment implementing these rules, but those of us who've drunk the Kool-Aid do it because we believe that there's value in standardized code.  Jamming something this trivial this down our throats when we've invested weeks if not months of man-hours to get compliant is just bad faith.

Just to weigh in my opinion, I would prefer to see this rule disabled by default, and the opposite rule suggested in #1121 should be implemented and enabled by default. Primarily this is to be consistent with tools like ReSharper which flags trailing commas as redundant and suggests they be removed.
Additionally, I think this rule only improves edit-ability slightly (and tools like ReSharper mitigate most of the complaints), whilst readability is impacted (albeit only slightly as well). I think its generally accepted that code is read many more times than its written, so we shouldn't promote write-ability at the sake of readability.
I've been following this project for some time, but not dipped my toes into actually helping yet. I would be happy to start here if there are no objections, although I may need a few pointers to get started :)

@berkeleybross At this point I would not expect the default behavior of this rule to change. A specific proposal to add an option (with values require/allow/omit like seen in #2050) might be entertained as long as the default is require.

Chipping in here as I was very surprised to discover this rule existed. A trailing comma in a list is a syntactic abberation that I would expect most developers (and English language speakers) to object to and actively remove. I implore you to reconsider it.

"but the new rule may also be disabled either due to personal preference or for any other reason."
I don't know if others see this as black-and-white as I do, but the minute you disable any of the rules for 'personal' reasons, you open up a can-of-worms.  Why should the lead developer get to chose what rules he likes and doesn't like?  Let's have a committee where we'll discuss which rules we're going to adopt as our standard.  Wait, the build engineering group also has a list of preferred rules, let's have a meeting to discuss which ones we're both going to use.
FWIW, If I allow my developers to select personal preferences, there's not a whole lot of benefit to using StyleCop if I've got several 'standards' floating around the shop.

We really hated this rule.


[High prior] GitHub Pages for prototyp gets error

@idrice24

This is a desaster:   before  push some thing  you have to make sure that it  works!!!

Checklist  of definition of done backend  and fontend shoul be done and add it to readme .md file

Checklist of frontend

Synchronized  with master  - successful ?
Make your change
Run a web api server - successful ?
Run start script - successful ?
Run deploy script - successful ?
Commit with issue number on comment - Create an issue if not exist
Push your change
Test all on cloud

NB:

I list  script in package.json file
Command is :  npm run  from angular-mns folder
YOU DO NOT have to change something in docs folder: files gets SHA , change is done with script only

now the project run successfully!!!


clean up instructions

Module: LAB_04L01_Implement_and_manage_AVD_profiles_AADDS
Lab/Demo: 00
Task: 00
Step: 00
Description of issue: There should be resource clean up instructions at the end of this lab as this is the last lab of "Azure AD DS" and there is no further lab in this course which depends on the output of this lab.

updated

thanks, I submitted four issues in AZ 304 lab instructions too but no response yet.


Navigation animation disappears in the whole app

Very nice lib, but I discovered a very rare behaviour.
I am using GTScrollNavigationBar with tableview.  Any ideas?

@l-lemesev: I could not replicate this bug on version 0.2. I guess this is the old bug. Please reopen if you can replicate it. Thanks.


fix CI

Recent commits were seen with unstable CI results.
If there is unstable test case, then mark them skippable.

Currently some test fails randomly:

it may fail on windows but passes all other platforms
it may fail a particular go version but passes all other versions

Example:

This affects the quality of the unit tests.

The trick is to detect output in stdout: panic: test timed out after


remember  breakpoint

I really need node-inspector remember breakpoints.

This would solve a huge pain point.  I'd imagine this could be accomplished with localStorage?

I think this is fixable, as is the issue with the constant 'prompt' about "enabling debugging on the scripts panel" - in fact, you should never, ever see this prompter.  What else are you doing with node-inspector!!! :-)
The basic idea is to save this info in localStorage associated with the browser session.  eg, localStorage for origin "http://<IP_ADDRESS>:8080".  Now, if it turns out that I decide I want to start running on port 8081, all my "settings" are gone, but ... not a huge issue.

I think this would be feasible through a chrome extension that opens the debugger, hooks into the event when a breakpoint is set, and saves it in offline storage.


Fix extensions/v1beta1/Ingress's replacement

The new API version should be networking.k8s.io/v1 as networking.k8s.io/v1beta1 is removed in 1.22 as well

This change made it in with another PR.


fix(Where): remove side effects from build

Previously, calling build() on a where clause would add more parameters each time. This side
effect has been moved so parameters are created in the constructor.
Closes #42

:tada: This PR is included in version 3.5.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket: