Fix IDA Segfaulting Bug in Magic Sync

Background
During defcon finals 22 the binary ni_visor was used for Magic Sync from various users. During this, someone introduced something into their local branch that caused a segfault. This started mutating the more people Magic Sync, which we came to call the @zardus Virus.
After a few minutes when anyone hit Magic Sync, that IDA would segfault while rapidly making changes. Attached is the binary. The BinSync repo can be found at:
https://github.com/Shellphish-DC22/f22-nivisor
If you connect then hit Magic Sync with @zardus as the priority it will segfault IDA. I've also attached the log right before this crash for reference.
20220822-151018.bs.log
nivisor.zip

Update
So I tried to debug this for a bit tonight and I've run out of time. Attached is an updated log with important information.
The important line is the last line before crash:
Filling function header to <FuncHeader: long main(args=3); @0x15316> now...

Essentially it crashes while filling the header of main with sane-looking types and args.
Culprit function is here:
https://github.com/binsync/binsync/blob/d50084e387412010066cf698b8f3debf008cb712/plugins/ida_binsync/ida_binsync/compat.py#L233
The thing that triggers the crash at this point actually has nothing to do with setting any values. Instead, the crash is triggered by calling refresh_pseudocode_view with the same address twice. It does not matter if you make changes before each of these calls.
This makes me thing that we are grabbing a reference to vu in a bad way. I'm really not sure, though. It's important to note that when I disabled this whole function, I still got crashes when simply decompiling other functions in this magic merge.
If looking to debug in the future, set breakpoints for the culprit function like so:
if addr == 0x3fc2:
   import remote_pdb; remote_pdb.RemotePdb("localhost", 4444).set_trace()

I recommend the next thing that we investigate is a better way to get a reference or look into why this one crashes more and others don't. It honestly seems random since all the things set in this function look sane. The function type gets changed to long and the args get new names. That is all. Something wonky is going on with references, I think.
Worst case scenario: we may need to disable magic sync for a bit :(, since the issue is not reproducible when doing only a single sync. Maybe look into closing code views?

just guessing here, but it seems like the only thing that happens between the two calls to refresh_pseudocode_view is:
https://github.com/binsync/binsync/blob/d50084e387412010066cf698b8f3debf008cb712/plugins/ida_binsync/ida_binsync/compat.py#L278-L280
this api call might be accessing the vdui_t.cfuncptr and related stuff after it might've gone out of scope due to vdui_t.refresh_view.
Python>v.cfunc, v.cfunc.lvars
(<ida_hexrays.cfuncptr_t; proxy of <Swig Object of type 'qrefcnt_t< cfunc_t > *' at 0x7faaef72d9b0> >, <ida_hexrays.lvars_t; proxy of <Swig Object of type 'lvars_t *' at 0x7faaef72d680> >)

Python>v.refresh_view(False)

Python>v.cfunc, v.cfunc.lvars
(<ida_hexrays.cfuncptr_t; proxy of <Swig Object of type 'qrefcnt_t< cfunc_t > *' at 0x7faaef72da70> >, <ida_hexrays.lvars_t; proxy of <Swig Object of type 'lvars_t *' at 0x7faaef72db90> >)

if you have the variable name and the address already, you can locate it by name or w/ vdui using ida_hexrays.locate_lvar in order to rename it or change its types with something like:
        @utils.multicase(ea=six.integer_types, lvar=ida_hexrays.lvar_t, string=(tuple, types.string))
        @classmethod
        def name(cls, ea, lvar, string):
            '''rename `lvar` to `string`'''
            lvarinfo = ida_hexrays.lvar_saved_info_t()
            if not ida_hexrays.locate_lvar(lvarinfo.ll, ea, lvar.name):
                raise exceptions.DisassemblerError

            res, lvarinfo.name = lvar.name, string

            if not ida_hexrays.modify_user_lvar_info(ea, ida_hexrays.MLI_NAME, lvarinfo):
                raise exceptions.DisassemblerError
            return utils.string.of(res)

        @utils.multicase(ea=six.integer_types, lvar=ida_hexrays.lvar_t, info=idaapi.tinfo_t)
        @classmethod
        def type(cls, ea, lvar, info):
            '''modify type of `lvar` to `info`'''
            if not lvar.accepts_type(info, False):
                raise ValueError("type ({:s}) does not fit".format("{!s}".format(info)))

            lvarinfo = ida_hexrays.lvar_saved_info_t()
            if not ida_hexrays.locate_lvar(lvarinfo.ll, ea, lvar.name):
                raise exceptions.DisassemblerError

            res, lvarinfo.type = lvar.type(), info

            if not ida_hexrays.modify_user_lvar_info(ea, ida_hexrays.MLI_TYPE, lvarinfo):
                raise exceptions.DisassemblerError
            return res

since the refresh_pseudocode_view func seems to be being used here for updating all of the views with any name or type changes, if possible, it might be safer to pull the refresh_pseudocode_view out of the for-loop that it's being used in. this way you do all the modifications to the the lvars first, and then refresh once afterwards.

@arizvisa So get this, I fixed one segfault be removing extra referenced to cfunc when accessing a vu ptr.
This line for instance:
https://github.com/binsync/binsync/blob/d50084e387412010066cf698b8f3debf008cb712/plugins/ida_binsync/ida_binsync/compat.py#L235
Creates a new cptr to cfunc (different address), which, when updating it, causes a segfualt because I assume its a UAF of some sort. I fixed it by removing all references that are created from the original object, and instead just used the object.
I've now hit an even more obscure crash.
Since discovering the first bug I pretty much have only been using vu.refresh_view directly, instead of iterating all views.
If you go here:
https://github.com/binsync/binsync/blob/d50084e387412010066cf698b8f3debf008cb712/plugins/ida_binsync/ida_binsync/controller.py#L225
This will get you a pointer to a vu of the current open window. All you need to do is just refresh_view(True), with no edits to the view, and boom, crash.

I'll try and examine closer in detail tomorrow (it's like midnight here), but I'd definitely verify your notifications/hooks if those are the the only actions that are causing it.
My latest refactor for my own proj is titled "persistence-refactor", because a ton of Hex-Rays types can go out-of-scope at the same time the user is using them, mostly things off of that vu object.. but even mblock_t and other things from the microcode. I've also had to differentiate between cfuncptr_t and cfunc_t as you can treat them the same, but the name points out that they're referenced differently.

Okay, had a chance to spend some time on it today. Didn't figure it out entirely...and ironically it took more time writing this down, heh.
Just so we're (sorta) on the same page, I'm running on 8.0.220729/Linux. After triggering the original crash, I noticed that the sync was pretty much only crashing on functions that I currently had open in another view within the decompiler. So, I rewrote both of the following functions so that the vu.refresh_view is moved outside the loop which guarantees that they're only called once per func_addr as perhaps that'd reduce the risk of a view being refreshed more than once in succession for the same function address.
https://github.com/binsync/binsync/blob/d50084e387412010066cf698b8f3debf008cb712/plugins/ida_binsync/ida_binsync/compat.py#L698
https://github.com/binsync/binsync/blob/d50084e387412010066cf698b8f3debf008cb712/plugins/ida_binsync/ida_binsync/hooks.py#L632
I also tweaked this next method so that it consolidates accessing that vdui_t.cfunc_t from the ida_func_code_view arg as per the original discussion. I didn't want to rewrite it, but just packed the related logic closer together in order to check if it was the culprit.
https://github.com/binsync/binsync/blob/d50084e387412010066cf698b8f3debf008cb712/plugins/ida_binsync/ida_binsync/compat.py#L233
Also added just some ways to notify which hooks are being called and when (just to be certain the issues are not hook-related).
def f(descr):
 def c(*args):
  print('got', descr, args)
 return c
import hook
hook.hx.add(ida_hexrays.hxe_print_func, f('hxe_print_func'))
hook.hx.add(ida_hexrays.hxe_func_printed, f('hxe_func_printed'))
hook.hexrays.add('refresh_pseudocode', f('refresh_pseudocode'))
hook.hexrays.add('switch_pseudocode', f('switch_pseudocode'))
hook.hexrays.add('open_pseudocode', f('open_pseudocode'))
hook.hexrays.add('close_pseudocode', f('close_pseudocode'))

Essentially these are my logs for the INTERRs that I got during the sync. This just confirms your "obscure crash" that acquire_pseudocode_vdui seems to trigger in-between accessing the microcode from the idb and invalidating it.
INFO | 2023-01-02 16:31:58,836 | binsync.common.controller | Successfully synced new changes from user for <Function: None read_0(args=0); @0x2448 vars=2 len=0x1f6>
acquire_pseudocode_vdui
265E: restored microcode from idb
265E: INTERR 50719
265E: deleted stale pseudocode from idb
got switch_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f11ba2e0> >,)
got hxe_print_func (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f11ba160> >, <ida_hexrays.vc_printer_t; proxy of <Swig Object of type 'vc_printer_t *' at 0x7f87f11ba250> >)
got hxe_func_printed (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f11ba010> >,)
got refresh_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f11ba2e0> >,)
acquire_pseudocode_vdui(done)
set_function_header(vdui,access)
set_function_header(vdui,we,are,good)
...
INFO | 2023-01-02 16:32:12,426 | binsync.common.controller | Successfully synced new changes from user for <Function: None main_state_set_rootpath(args=0); @0x15762 vars=0 len=0x1f>
acquire_pseudocode_vdui
3966: restored microcode from idb
3966: INTERR 50719
3966: deleted stale pseudocode from idb
got switch_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f1550240> >,)
got hxe_print_func (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f156f6f0> >, <ida_hexrays.vc_printer_t; proxy of <Swig Object of type 'vc_printer_t *' at 0x7f87f1550570> >)
got hxe_func_printed (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f156f6f0> >,)
got refresh_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f156f6f0> >,)
acquire_pseudocode_vdui(done)
set_function_header(vdui,access)
set_function_header(vdui,we,are,good)
got hxe_print_func (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f1550e40> >, <ida_hexrays.vc_printer_t; proxy of <Swig Object of type 'vc_printer_t *' at 0x7f87f1551470> >)
got hxe_func_printed (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f1550ff0> >,)
got hxe_print_func (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f1550e40> >,
...
INFO | 2023-01-02 16:32:13,864 | binsync.common.controller | Successfully synced new changes from user for <Function: None constr_11(args=0); @0x8975 vars=0 len=0x12>
acquire_pseudocode_vdui
297C: restored microcode from idb
297C: INTERR 50719
297C: deleted stale pseudocode from idb
got switch_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f1551d40> >,)
got hxe_print_func (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f1551c20> >, <ida_hexrays.vc_printer_t; proxy of <Swig Object of type 'vc_printer_t *' at 0x7f87f15507e0> >)
got hxe_func_printed (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f15519e0> >,)
got refresh_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f1551d40> >,)
acquire_pseudocode_vdui(done)
set_function_header(vdui,access)
set_function_header(vdui,we,are,good)
...
INFO | 2023-01-02 16:32:18,547 | binsync.common.controller | Successfully synced new changes from user for <Function: long sub_39BA(args=1); @0x39ba vars=1 len=0xba>
acquire_pseudocode_vdui
39BA: INTERR 50893
3FC2: deleted stale microcode from idb
3FC2: INTERR 51551
INFO | 2023-01-02 16:32:18,747 | binsync.common.controller | Banishing exception: Decompilation failed: 3fc2: INTERR: 51551
acquire_pseudocode_vdui
87C4: restored microcode from idb
87C4: restored pseudocode from idb
got switch_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f15ddbf0> >,)
got hxe_print_func (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f15ddb30> >, <ida_hexrays.vc_printer_t; proxy of <Swig Object of type 'vc_printer_t *' at 0x7f87f15dd5c0> >)
got hxe_func_printed (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f15dd4d0> >,)
got refresh_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f15ddbf0> >,)
acquire_pseudocode_vdui(done)
set_function_header(vdui,access)
set_function_header(vdui,we,are,good)
...
INFO | 2023-01-02 16:32:18,547 | binsync.common.controller | Successfully synced new changes from user for <Function: long sub_39BA(args=1); @0x39ba vars=1 len=0xba>
acquire_pseudocode_vdui
39BA: INTERR 50893
3FC2: deleted stale microcode from idb
3FC2: INTERR 51551
INFO | 2023-01-02 16:32:18,747 | binsync.common.controller | Banishing exception: Decompilation failed: 3fc2: INTERR: 51551
acquire_pseudocode_vdui
87C4: restored microcode from idb
87C4: restored pseudocode from idb
got switch_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f15ddbf0> >,)
got hxe_print_func (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f15ddb30> >, <ida_hexrays.vc_printer_t; proxy of <Swig Object of type 'vc_printer_t *' at 0x7f87f15dd5c0> >)
got hxe_func_printed (<ida_hexrays.cfunc_t; proxy of <Swig Object of type 'cfunc_t *' at 0x7f87f15dd4d0> >,)
got refresh_pseudocode (<ida_hexrays.vdui_t; proxy of <Swig Object of type 'vdui_t *' at 0x7f87f15ddbf0> >,)
acquire_pseudocode_vdui(done)
set_function_header(vdui,access)
set_function_header(vdui,we,are,good)

According to https://hex-rays.com/products/decompiler/manual/sdk/structvdui__t.shtml#a44e1923f1e01bed28bf52f862520052d, this might only happen when its parameter is True. Also, refresh_ctext claims it refreshes all open pseudocode windows for an address. I'm not sure if refresh_view depends on it, but you might be able to avoid iterating through all pseudocode windows since you only need to reference them by address for a refresh.
So, it doesn't seem to be related to events/hooks at all like I suggested...and I've actually had this error happen outside of magic-sync in between testing. If you end up also encountering it outside magic-sync, I'd consider triaging it as a different issue...and maybe you can ping Hexrays on these error codes. They're usually pretty quick to respond.

Update

I fixed one crash in #231, but one more still remains. I'll likely close this issue and open a new one with the second crash. It seems it may be best to contact HexRays for this.

Not sure if it's the same crash that I'm getting, as since I've upgraded my system over the weekend Qt has been a little unstable...or at least the Lucid plugin crashes w/ a null deref.
Anyways, for me magic sync is crashing in the middle of an ida_hexrays.decompile_func while trying to access type info via get_tinfo_size. It's happening in response to an ida_kernwin.execute_sync call as per another thread. If it's the same one that you're encountering, it could be one of those functions decorated with execute_write. I'll keep an eye out for the new issue, though.

Hmmm that's a different crash than I had before... I'll have to check again. As far as execute_sync goes, I thought the whole point of those functions is that they can be called from many threads to queue for access on the Mainthread. Am I interpreting this wrong?
------- Original Message -------
On Monday, February 13th, 2023 at 12:45 PM, Ali Rizvi-Santiago @.***> wrote:

Not sure if it's the same crash that I'm getting, as since I've upgraded my system over the weekend Qt has been a little unstable...or at least the Lucid plugin crashes w/ a null deref.
Anyways, for me magic sync is crashing in the middle of an ida_hexrays.decompile_func while trying to access type info via get_tinfo_size. It's happening in response to an ida_kernwin.execute_sync call as per another thread. If it's the same one that you're encountering, it could be one of those functions decorated with execute_write. I'll keep an eye out for the new issue, though.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

I thought the whole point of those functions is that they can be called from many threads to queue for access on the Mainthread.
Am I interpreting this wrong?

Nah, you're correct.
The path that my system crashed on was from execute_sync which was only called from functions decorated by @execute_write. So for me, this following ida_hexrays.decompile was the only candidate that matched what my crash was showing (I'm also on linux).
@execute_write
def get_func_stack_var_info(func_addr) -> typing.Dict[int, StackVariable]:
    try:
        decompilation = ida_hexrays.decompile(func_addr)
    except ida_hexrays.DecompilationFailure:
...

I need to upgrade to IDA 8.2, though, as I'm still on 8.0. So this specific crash might be bunk (because y'know, software sucks). In your next issue, though, let me know what crash you're getting and I'll see if I can repro on my end.

This bug has brought me nothing but pain. So I discovered something interesting. If you go to this line: https://github.com/binsync/binsync/blob/603ca75a303212b7eeb16b9645650822fe8f4a7a/binsync/common/controller.py#L825
Then add the following:
                if isinstance(pref_art, Function) and pref_art.name != "sub_3D26":
                    continue

The bug can be reproduced in such a way that IDA does not segfault out, but it has internal errors that try to force you to send an email to HexRays. What this little change in code does is avoid syncing any functions that are not the one function I know crashes while also allowing structs. What I've sussed out from this is that structs being synced plus this function is causing the crash... which just makes no sense.

@arizvisa I've found the bug all the way at the bottom... and it's a little wild. I thought you'd care about it since it's so strange and funny.
PoC
To reproduce database corruption in IDA first download this
nivisor_crash.zip zip file. The ziped folder contains the nivisor binary and the poc script poc.py. The script does not require you to have anything installed. It's only requirement is that it's running in IDA (no BinSync needed).
Now do the following precisely:

Open the binary nivisor in IDA, don't change the function IDA is focused on by default
Run the script file poc.py
You should now be looking at the decompilation of start
Hit F5 (or just redecompile the current function).
Close IDA, pack and don't save database
Your database is now corrupted (should have a popup).

For reference, all the script does is set the following struct in IDA:
    bad_struct = {
        "name": "gcc_va_list",
        "size": 16,
        "members": {
            0: {
                "member_name": "gp_offset",
                "offset": 0,
                "type": "unsigned int",
                "size": 4
            },
            4: {
                "member_name": "fp_offset",
                "offset": 4,
                "type": "unsigned int",
                "size": 4
            },
            8: {
                "member_name": "overflow_arg_area",
                "offset": 8,
                "type": "",
                "size": 8
            },
        }
    }

The Fix
For now, it's just a hotfix that is dumb in #236. Never set a struct with this name. In some form, this will be reported to the IDA Pro devs in the next few days.


Test that no superglobal is read while creating a server request

I'm not sure if this is the desired behavior, but with these tests we can check that the creation of a new server request is not affected by global state.

What about @backupGlobals annotation? Does that help here?

After review of PHPUnit documentation, it looks that, by default, each test is isolated from superglobal mutation, so there is no need to manually backup their values.

In that case, is this change still viable?

Yes, I've updated the PR to take advantage of that feature.

Thanks!


cant install azure-cognitiveservices-speech

Bug
I am unable to install  azure-cognitiveservices-speech on my 64 bit windows machine.
The error I am getting :
>pip install azure-cognitiveservices-speech
ERROR: Could not find a version that satisfies the requirement azure-cognitiveservices-speech (from versions: none)
ERROR: No matching distribution found for azure-cognitiveservices-speech

Information about my python interpreter:
>python
Python 3.9.0 (tags/v3.9.0:9cf6752, Oct  5 2020, 15:34:40) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.

I also tried to install it using .whl files I found here
I tried to install 8 .whl files that are intended for windows  , but all failed .
I also tried to install on a different 64 bit Windows 10 OS machine , but it failed there as well

It was a duplicates issue  ,  my apologies


webpack-dev-server crashes quietly on sass @import in component

I have one line that makes the difference between webpack-dev-server crashing without any error message and running smoothly. The only thing that happens is that it does not report stats, however it does print: "Webpack development server listening on localhost:3001".
The line in question is
// file: /components/recentlyReported.scss

@import "../variablesAndMixins.scss";

which is imported like other scss files in the example, like so:
// file: /components/recentlyReported.js

if (__CLIENT__) {
  require('./RecentlyReported.scss');
}

It seems to work fine for files in /views, however the component in question is in /compoents, and is used as a sub component of my Dashboard view. When I uncomment the @import line, start webpack-dev-server, then uncomment the @import line, it works fine. It is only initially that it does not work.
Why does this matter? and how can I enable error messages for things like this?

I was fighting with this exact problem all day yesterday. It must be some problem with sass-loader. If anyone can figure this out, please post something here.

Do you have an alternative way to share variables across .scss files imported this way?

I don't, no. :disappointed:

FWIW, your issue is probably sass/node-sass#857.
TL;DR: node-sass hangs when importing a file from many other files. It seems that increasing the environment variable UV_THREADPOOL_SIZE for the host of node-sass usually fixes the deadlock.
People generally just seem to be prefixing their webpack command with env UV_THREADPOOL_SIZE=100; I can report that that has worked for me, as well.


object return by client-go informer don't have  api Kind info

When I use informer to watch cluster resource infos, I find  runtime.Object return by informer  don't include object Kind info.
fmt.Println(obj.GetObjectKind().GetVersionKind().String()) print  /, Kind=

I don't think the Kind is returned either with List

The k8s REST API does not provide the Kind value. kubectl (or one of its dependencies) inserts the Kind information. Perhaps the code that figures the Kind info can be moved to client-go? Just for example, see get cmd and how it inserts the List value.

This occurs because the gvk is cleared during decoding: https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apimachinery/pkg/runtime/serializer/versioning/versioning.go#L256.
This is intentional. See https://github.com/kubernetes/kubernetes/pull/59264#issuecomment-362575608 for more details.

Reported in kubernetes/kubernetes#3030
Fix in progress in kubernetes/kubernetes#63972

Thank you for the update @nikhita!

/reopen

For those of us still looking for a concise way to add TypeMeta information to runtime objects, here you go!
// addTypeInformationToObject adds TypeMeta information to a runtime.Object based upon the loaded scheme.Scheme
// inspired by: https://github.com/kubernetes/cli-runtime/blob/v0.19.2/pkg/printers/typesetter.go#L41
func addTypeInformationToObject(obj runtime.Object) error {
    gvks, _, err := scheme.Scheme.ObjectKinds(obj)
    if err != nil {
        return fmt.Errorf("missing apiVersion or kind and cannot assign it; %w", err)
    }

    for _, gvk := range gvks {
        if len(gvk.Kind) == 0 {
            continue
        }
        if len(gvk.Version) == 0 || gvk.Version == runtime.APIVersionInternal {
            continue
        }
        obj.GetObjectKind().SetGroupVersionKind(gvk)
        break
    }

    return nil
}

Example what everyone needs to do in order to use owner reference with informers and client-go
https://github.com/zalando-incubator/stackset-controller/blob/ec093f68f18c7c21ae03e44a14666fc6e8d4a5d3/controller/stackset.go#L1084-L1096

/remove-lifecycle rotten

/remove-lifecycle stale

/remove-lifecycle rotten

/important-longterm

/remove-lifecycle rotten

/remove-lifecycle rotten


Per-app compression option

Long story short
Currently, there seems to be no way to enable compression for methods like json_response, while it'd be especially beneficial for performance to compress big JSON responses. While it's possible to compress JSON responses using the low-level API, one has to jump through a few hoops to do so. I'd imagine a app-wide switch to enable opportunistic compression on all responses.
Expected behaviour
app.enable_compression(force=None)

I don't think it's good idea. Enabling something globally may lead to confusing behavior very easy.
Moreover, adding compression parameter to Response ctor and json_response function to use it on per-request basis doesn't make much sense.
Compression should be controlled by client.
If client informs server about accepted compression via Accept-Encoding header aiohttp web server applies this compression automatically.

Compression should be controlled by client.

Yes exactly that's what I mean by opportunistic, but to my knowledge, aiohttp never compresses responses unless .enableCompression() is called on the response, even in the presence of supported Accept-Encoding request headers.

Enabling something globally may lead to confusing behavior very easy

I think it's reasonably to enable it globally if the negotiation with the client works reliably. For comparison, in the Express framework, I can simply app.use(compression()), and the library handles the rest.

is there value to do compression on aiohttp side? usually nginx is in front of application.

While I generally agree and in fact do run most of my HTTP servers behind a reverse proxy, having resonable performance standalone is still valuable to me.

Digging into the subject deeper I'm inclining to don't implement global compression in aiohttp.
The reason is: no clean rule for deciding where do compression and where don't.
Say, NGINX has complex options for controlling compression -- and these options are not shrinked to simple boolean parameter.
From other hand the issue could be solved by custom middleware (with settings and behavior which are satisfactory for concrete use case).


Pretty printer enhancements

Spacing

[ ] remove trailing whitespace
[ ] Remove these annoying spaces before dots

CLASS zfh_class_two DEFINITION
  PUBLIC
  FINAL
  CREATE PUBLIC .

try .

    catch cx_root into cx.
  endtry.

    INTERFACES if_t100_dyn_msg .

Methods

[ ] Align method definitions

    METHODS error
      IMPORTING
        param_first   TYPE REF TO data
        param_other   TYPE REF TO zfh_param_pretty_print OPTIONAL
      EXPORTING
        param_ex      TYPE REF TO data
      CHANGING
        param_ch      TYPE REF TO data
      RETURNING VALUE(result) TYPE REF TO zfh_param_pretty_print
      EXCEPTIONS cx_root.

METHODS error
  IMPORTING param_first   TYPE REF TO data
  RETURNING VALUE(result) TYPE REF TO zfh_param_pretty_print.

Most often used expressions

[ ] Align components in value constructor

DATA(key) = VALUE structure_type(
  aadsadas = apple
  basdsad = tree
  cqwewqeqew = cl_x=>y ).

[ ] Cond/switch
either

WHEN cl_x=>y
 THEN NEW cl_foo( )

WHEN cl_y=>z
 THEN  NEW cl_bar( )

or
WHEN cl_x=>y
THEN NEW cl_foo( )

WHEN cl_y=>z
THEN  NEW cl_bar( )

Remove redundant parentheses

based on setting

IF( x = 5 ).
ENDIF.

IF x = 5.
ENDIF.

for whitespace also see https://github.com/abaplint/abaplint/issues/483
redundant parentheses, see https://github.com/abaplint/abaplint/issues/654


Generate data clusters

All Submissions Basics:
#65

[x] Have you followed the guidelines in our Contributing document?
[x] Have you checked to ensure there aren't other open Pull Requests for the same update/change?
[x] Have you checked all Issues to tie the PR to a specific one?

All Submissions Cores:

[x] Have you added an explanation of what your changes do and why you'd like us to include them?
[x] Have you written new tests for your core changes, as applicable?
[x] Have you successfully ran tests with your changes locally?
[x] Does your submission pass tests, including CircleCI, Travis CI, and AppVeyor?
[x] Does your submission have appropriate code coverage? The cutoff threshold is 95% by Coversall.

New Model Submissions:

[x] Have you created a .py in ~/pyod/models/?
[x] Have you created a _example.py in ~/examples/?
[x] Have you created a test_.py in ~/pyod/test/?
[x] Have you lint your code locally prior to submission?

Brief Description
This algorithm generates one (or many) clusters of data points with different/same sizes and densities based on the user's choice passed by the parameters.
It generates the required ratio of outliers controlled by the contamination parameter and distributes them on the clusters.
It avails of the make_blobs function provided by sklearn to create the clusters; and main part of the algorithm is to maintain and validate the consistency of the data splits among the different clusters.
It is very well documented, and I believe if you read the documentation (comments) you will get it easily.

As per previously mentioned in the related issue #66 , having different clusters of data with different sizes and densities makes outliers detection challengeable  especially for those type of algorithms that based of k-nearest neighbors such as LOF , LDOF, LoOP, HiCS and SOD and others.

Pull Request Test Coverage Report for Build 822

108 of 132 (81.82%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.5%) to 94.834%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pyod/utils/data.py
69
93
74.19%

Totals

Change from base Build 819:
-0.5%

Covered Lines:
3635

Relevant Lines:
3833

💛  - Coveralls

Pull Request Test Coverage Report for Build 822

108 of 132 (81.82%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.5%) to 94.834%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pyod/utils/data.py
69
93
74.19%

Totals

Change from base Build 819:
-0.5%

Covered Lines:
3635

Relevant Lines:
3833

💛  - Coveralls

Pull Request Test Coverage Report for Build 822

108 of 132 (81.82%)  changed or added relevant lines in 2 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.5%) to 94.834%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

pyod/utils/data.py
69
93
74.19%

Totals

Change from base Build 819:
-0.5%

Covered Lines:
3635

Relevant Lines:
3833

💛  - Coveralls

@yzhao062
I did all tests required and more, even I added a test missed in for check_consistent_shape function. Yet it is still not happy.
Would you please fix/lower the threshold of coveralls for this one?
Thanks.

Do not worry about that. I will take care that thing :)

I will do some necessary cleanup work and this function should be available in V.0.7.0. :)


Is the configuration name blocked process threshold correct?

When I look st sys.configurations, it has 'blocked process threshold (s)', not 'blocked process threshold'.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 06ed8fd8-016f-7779-ed08-462f505c5f3e
Version Independent ID: af26e293-4d4c-3810-b350-6dfab61a6161
Content: blocked process threshold Server Configuration Option - SQL Server
Content Source: docs/database-engine/configure-windows/blocked-process-threshold-server-configuration-option.md
Product: sql
Technology: configuration
GitHub Login: @MikeRayMSFT
Microsoft Alias: mikeray

@iampeterdaniels -- Peter, thank you for your question.  You may find answers here:

DBA Stack Exchange
Stack Overflow

You are welcome to submit a bug a here:

SQL Server Feedback


Ticket Validation Failure

Hi to all,
I've a problem during the ticket validation: when the shib-cas-authn call the cas/serviceValidate I obtain an error. The log is:
<cas:serviceResponse xmlns:cas='http://www.yale.edu/tp/cas'>
<cas:authenticationFailure code='INVALID_TICKET'>
Il ticket ''{0}'' non � stato riconosciuto
</cas:authenticationFailure>
</cas:serviceResponse>
But if I look at the CAS logs, I obtain that the service ticket has been correctly validated. So, what is the problem?
Thank you in advance.

Hello,
Did you find a solution for this problem? I have the same behaviour

I don't remember it exactly, but you should try to force the "renew" parameter to true value to bypass the problem.

We identify a similar problem with IdP 3.4 and CAS 6.0 with shib-cas-authn3. In our case, empty attributes sent by CAS to Shib-cas-auth3 produce "InvalidTIcket" errors and it breaks authentication workflow.


build_docs does not overwrite top bar config files

Hi,
After built with nbdev_build_docs,  for a given settings.ini file, I modify the user (in the settings.ini)
and I build the docs again with nbdev_build_docs. The top bar of the documentation keeps pointing to the first user  home and GitHub URLs.
This simple solution has worked for me:
I removed the conditional that prevents the overwriting of the configuration files.
Thanks for this great package!

No, we don't want to overwrite the templates since they are default provided to users, and users may want to change them. If we take your change, those customizations would be erased at each new build.

Oh!, I see your point, thanks!


Static measured boot

This card is to track static measured boot support and how to make offline encryption depend on it to achieve #119 in case of a compromised or tampered OS image.
In order to support static measured boot, we will rely on the UKI implementation given from #347 and the lifecycle manager in #868 .
This card depend on #871.

Blocked by https://github.com/kairos-io/kairos/issues/868

Blocked by #1152

This is done


Allow setting of minimal velocity

Hi,
is it possible to implement a minimal velocity? We are using the teb_local_planner with success on either simulated or real ackermann driven robots.
But in some cases we are facing the problem that the robot base cannot execute the desired velocity.
E.g. it would only start driving at a minimal velocity of 0.1 m/s.
Best regards and thumbs up for the great work!

Hi @balamesh,
in general, adding a lower bound on the minimum velocity should be possible. However, integrating this into the optimization scheme significantly increases complexity since we already support switching between forward and backwards motions  (it might introduce non-smooth cost functions or even more local minima).
In my opinion, adjusting the controls/velocities after obtaining the optimization result should work fine in many applications, e.g. using a simple three-point-controller before forwarding the velocities to the actual base controller:
if ( abs(v_out >= v_min) ) v_out = v;
else if ( abs(v_out >= eps) ) v_out = v_min;
else v_out = 0;

However, the threshold eps depends on

the maximum acceleration (the robot must be able to reach the subsequent discrete state of the local plan)
and the goal tolerance (the robot must be able to reach the desired goal region)

If v_min is small and eps is chosen appropriately, it should not influence stability significantly to my mind.
It would be really straight forward to add those few lines to the computeVelocityCommand() method in teb_local_planner_ros.cpp but in my opinion it is more suitable and intuitive if these platform dependent heuristics are included in the robot's base controller as long it is not subject to the actual optimization/planning scheme.

Yes thanks a lot for your help. We managed to solve this by implementing a minimum velocity on the controller side.
And sorry for the late response!


ci: move info filter to ci config

Locally seeing bazel info lines can be helpful for things like
--announce_rc. This should disable for CI only. Except
for jobs that set NO_BUILD_SETUP which isn't many.
Signed-off-by: Keith Smiley<EMAIL_ADDRESS>

/retest


make velocity module pub and the xr feature not default

makes the velocity public since the only component in there is public and that seems to be needed for networked remote players
also make the xr feature not a part of default features since it seems more a feature to enable if needed than disable manually

👍


Remove version: CrystalDewWorld.CrystalDiskInfo.ShizukuEdition version 8.17.5

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

Close with reason: Duplicate;


CoinDesk exchange rates no longer accurate

We've had multiple reports of the CoinDesk exchange rates being inaccurate, most recently in issue #304.
I can attest to this personally, as I've noticed that the exchange rate for COP (Colombian Peso) has been very inaccurate recently. For example, the other day:

coindesk: 1 USD = 4,219 COP
actual rate: 1 USD = 4,757 COP

The raw JSON from the coindesk feed contains this disclaimer:

"This data was produced from the CoinDesk Bitcoin Price Index (USD). Non-USD currency data converted using hourly conversion rate from openexchangerates.org"

However, if I pull directly from the source, I get very different exchange rates:

direct from CoinDesk API: 1 USD = 4,218.99 COP
direct from OpenExchangerRates API: 1 USD = 4,765.08 COP

So the OpenExchangeRates API has an accurate value, while CoinDesk does not. So if CoinDesk is indeed pulling from this source, then they're doing so poorly. It may be that they're no longer updating those rates...
Thus I went looking for alternative sources. The trouble is that most of the well-known sources for fiat exchange rates (e.g. OpenExchangeRates.org) are either not free, or their free service requires registration, and your account token is rate-limited (e.g. 1,000 API requests per month).
One possible solution to this would be to aggregate these exchange rates server-side, and provide an API to Phoenix clients. We can ponder the pros and cons of this approach later. For now, I've discovered an alternative solution: the Coinbase API
Coinbase's free & public API supports all the same currencies we were using Coindesk for... except certain countries that the USA is sanctioning:

CUP: Cuban Peso
ERN: Eritrean Nakfa
IRR: Iranian Rial
KPW: North Korean Won
SDG: Sudanese Pound
SOS: Somali Shilling
SYP: Syrian Pound

So what I've done is replace coindesk with coinbase for all the other fiat currencies. That means 142 currencies that were previously using Coindesk, and getting stale exchange rates, are now using coinbase and getting accurate exchange rates.
For the 7 currencies listed above... well... we're still using coindesk for them :man_shrugging:
Other technical notes:

I removed the "Maltese Lira", as they adopted the Euro back in January 2008
I added CNH, which is the Chinese Yuan (offshore market) rate. We had a request for this on Jitbit awhile back.
I also added CUP_FM, which is the Cuban Peso (free market) rate. From issue #298.

I'm also wondering if the new query [...] could have any impacts on the iOS notification-service extension?

I investigated this. It doesn't seem to cause problems. The memory limit is 24 MB, and we're using around 9.5 MB, with a high watermark of 14 MB. This seems to be the case whether we pull rates from coinbase or not.
I think the problem we had initially is that:

the CurrencyManager was updating ALL currencies
the CoinDesk API was creating 146 separate HTTPS connections

Now we disable auto-refresh in the CurrencyManager, and only refresh the target fiat currency. So that's a max of 2 HTTPS connections. (e.g. for COP, we need to hit coinbase for the COP/USD rate, and blockchain.info for the USD/BTC rate).
(Note that coinbase does have the USD/BTC and EUR/BTC rate. So we could theoretically drop blockchain.info, and just pull that info from coinbase.)

I think the about screen for the iOS app needs to be updated to mention the new sources.

Done in 9df320e


added #index action back into Sessions Controller.

I was receiving error #index action missing for Sessions Controller. The index action should be in the sessions controller because it's the action that loads when someone goes to load the sign in form.

Thank you for the pull request. The action that loads when someone goes to load the sign in form in the new not index. See here https://github.com/plataformatec/devise/blob/master/lib/devise/rails/routes.rb#L49

I understand. But the #index is getting loaded when I trace the steps using
pry debugger.
On Monday, June 19, 2017, Rafael França<EMAIL_ADDRESS>wrote:

Thank you for the pull request. The action that loads when someone goes to
load the sign in form in the new not index. See here https://github.com/
plataformatec/devise/blob/master/lib/devise/rails/routes.rb#L49
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/plataformatec/devise/pull/4563#issuecomment-309508119,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEXKPHV36Zwkwii5IVw-PHf1IJNGqDBrks5sFq45gaJpZM4N9qf-
.

Is your application overriding the devise controller somehow?  Can you please provide a sample application that reproduces the error?

SessionsController shouldn't have an index action. If you're getting an error about this, please open an issue so we can discuss what's happening.
Thanks.


Font for PowerLine in terminal

VSCode Version:newest
OS Version: Os X
zsh

I have a problem to display font for PowerLine in terminal.
To solve this I had read this (https://github.com/Microsoft/vscode/issues/15119#issuecomment-259248159) and  I put this in my config file.
"terminal.integrated.fontFamily": "MYFONTNAME"
But actualy to find MYFONTNAME was pretty hard.
Here is my solution :

Open font Book.app
Select your font (for me it was "DejaVuSansMono Nerd Font Livre")
click on the "i" on the left part of the window

The MYFONTNAME is equal to PostScriptName.

I hope it could be help.

Created https://github.com/Microsoft/vscode-docs/issues/1534 to add a FAQ entry on the website.


Webpack > Error: Can't resolve

Hi,
I have an issue with your plugin and I don't really understand why. It looks like Webpack can't resolve the svg path for some reason.
Node version: 10.4.0
Webpack version: 4.16.0
iconfont-webpack-plugin: 3.0.0
Here's my folder structure:
├── src
│   ├── assets
│   │   ├── images
│   │   │   ├── icons
│   │   │   │   └── some-icon.svg
│   └── styles
│   │   │   ├── components
│   │   │   │   └── _some-file.scss

Here's my webpack config
const path = require('path');
const MiniCssExtractPlugin = require('mini-css-extract-plugin');
const IconfontWebpackPlugin = require('iconfont-webpack-plugin');

const isDev = process.env.NODE_ENV !== 'production';

module.exports = () => ({

  entry: [
    'eventsource-polyfill',
    'webpack-hot-middleware/client?reload=true',
    path.join(process.cwd(), 'src/scripts/main.js'),
  ],

  output: {
    path: path.resolve(process.cwd(), 'dist'),
    publicPath: '',
    filename: '[name].js',
    chunkFilename: '[name].chunk.js',
  },

  resolve: {
    modules: ['node_modules', 'src'],
    extensions: ['*', '.js', '.json'],
  },

  plugins: [
    new MiniCssExtractPlugin({
      filename: isDev ? 'styles/[name].css' : 'styles/[name].[hash].min.css',
      chunkFilename: isDev ? 'styles/[id].css' : 'styles/[id].[hash].css',
    }),
  ],

  module: {
    rules: [
      {
        test: /\.(sa|sc|c)ss$/,
        use: [
          isDev ? 'style-loader' : MiniCssExtractPlugin.loader,
          {
            loader: 'css-loader',
            options: {
              sourceMap: false,
            },
          }, {
            loader: 'postcss-loader',
            options: {
              plugins: loader => [
                require('autoprefixer'), // eslint-disable-line global-require
                new IconfontWebpackPlugin({
                  resolve: loader.resolve,
                  fontNamePrefix: 'purr-',
                }),
              ],
              sourceMap: false,
            },
          }, {
            loader: 'sass-loader',
            options: {
              includePaths: [path.resolve(__dirname, 'src', 'scss')],
              sourceMap: false,
            },
          },
        ],
      },
    ],
  },
});

My scss file:
.icon {
  font-icon: url('../../assets/images/icons/some-icon.svg');
}

The error I get:

You can see all the code in my repo inside the dev branch : https://github.com/Manoz/purr-starter/commit/56ff0e79b12ccc7015715566cade91bc83c56ac6
I've tested different path in the .scss file for the SVG just in case but looks like every path I try fail.
I'm not sure if it's related to your plugin or not :/

That's a problem of the sass loader.
The resolve-url-loader tries to fix it.


cmd/cue: help vet talks about non-existent -e flag

What version of CUE are you using (cue version)?

$ cue version
cue version +760aa115 linux/amd64

Does this issue reproduce with the latest release?
Yes
What did you do?
$ cue help vet
...
By default, each file is checked against the root of the loaded CUE files.
The -e can be used to only verify files against the result of an expression
evaluated within the CUE files. This can be useful if the CUE files contain
a set of definitions to pick from.

Examples:

  # Check files against a CUE file:
  cue vet foo.yaml foo.cue

  # Check files against a particular expression
  cue vet translations/*.yaml foo.cue -e Translation

If more than one expression is given, all must match all values.

Usage:
  cue vet [flags]

Flags:
  -c, --concrete                 require the evaluation to be concrete
  -h, --help                     help for vet
  -t, --inject stringArray       set the value of a tagged field
      --list                     concatenate multiple objects into a list
      --merge                    merge non-CUE files (default true)
  -n, --name string              glob filter for file names
  -p, --package string           package name for non-CUE files
  -l, --path stringArray         CUE expression for single path component
  -I, --proto_path stringArray   paths in which to search for imports
  -d, --schema string            expression to select schema for evaluating values in non-CUE files
      --with-context             import as object with contextual data

Global Flags:
  -E, --all-errors   print all available errors
  -i, --ignore       proceed in the presence of errors
  -s, --simplify     simplify output
      --strict       report errors for lossy mappings
      --trace        trace computation
  -v, --verbose      print information about progress

This should be a reference to -d - potentially with some rewording.
Raising this as an issue because it's related to #750

I reckon that should be -d.

This issue has been migrated to https://github.com/cue-lang/cue/issues/751.
For more details about CUE's migration to a new home, please see https://github.com/cue-lang/cue/issues/1078.


/hello/: Update hero CTA to be more visible.

This makes the button a solid white rather than a transparent button
with a white outline so that it will draw more attention to itself.

I'm not sure this is the best color for this button, but it's clearly an improvement, so merged, thanks @brockwhittaker!  I don't think we'll forget to keep iterating on this :)


Option to not require allow_url_fopen?

Any option to not require allow_url_fopen in php.ini?
Can curl be used as an alternative?

cURL can be used, you'd just have to create a Client Adapter for it.


Rebrand

Hi, looking forward to rebrand alpha-wallet for a small company. I want to change the splash screen, app name, app logo to that of the project. Want to know the exact files to change. i want to rebrand and make it open source

Reference: https://github.com/AlphaWallet/alpha-wallet-android/wiki/How-to-customize-splash-screen-and-app-logo%3F

thanks

My pleasure.

i want to add binance and polygon networks as locked chains. I know how to go about that but im a bit confused about the name i should add to the locked chains section. i tried Binance (BSC) but it didnt work

I know about it, wat i want to know is the type of string i should input.

The ids are constants, need class name to reference them.

ok i got it now thanks

You are welcome!


Add SQL statements to SaveChanges batch

Is it possible to add to the SQL batch generated for the .SaveChanges (or .SaveChangesAsync) method of DbContext?
Background:
I'm wanting to set entity navigation properties without loading the related entity. So I do something like:
class Client {
	int Id { get; set; }
	string Name { get; set; }
}

class Project {
	int Id { get; set; }
	string Name { get; set; }

	Client Client { get; set; }
}

async Task CreateProject(int clientId) {
	using(var context = new XYZDbContext()) {
		var client = new Client { Id = clientId };
		context.Clients.Attach(client);

		var project = new Project {
			Client = client,
		};

		context.Projects.Add(project);

		await context.SaveChangesAsync();
	}
}

However, if the attached Client doesn't exist, all I get is a SqlException wrapped in a DbUpdateException and I have to do string parsing of the SQL error message to determine the property that caused the error so that I can inform the user which input to correct.
What I'd like to do is to add to the SQL batch generated by .SaveChangesAsync to check for the existence of each of the referenced entities:
exec sp_executesql N'SET NOCOUNT ON;
-- Check that Client exists
IF NOT EXISTS (SELECT 1 FROM Clients WHERE Id = @p1)
	THROW 60000, ''Client does not exist'', 1

INSERT INTO [Projects] ([Id], [ClientId], [Name])
VALUES (@p0, @p1, @p2);
',N'@p0 int,@p1 int,@p2 nvarchar(4000)',@p0=1,@p1=6,@p2=N'Project 1'

This way I can customize the SQL messages and precisely determine the invalid input

Something like this would typically be handled with a stored procedure (tracked by #245). Another option would be to check for existence before saving, but that is obviously not part of the same batch. If you are feeling brave, you could look at overriding the internal components that execute batches... but that's not a documented scenario, so you'd be on your own in terms of figuring out how to do it.


create a laucher for the toolbox

[x] Create a script that will source a specific python bin and library to run the toolbox
[x] Replace the simlink with the launcher
[x] rewrite the installation tool to reflect that
[x] make use of .sctrc possible
[x] display SCT version in the "echo" (get it in version.txt)

Ready to be tested by other!

Ready to be tested by other!

Did not account for testing directories and scripts

The -getorient issue (issue #765) may be responsible for the fail of other test: for ex. if the function get an image orientation in command line it will fail. (See https://travis-ci.org/neuropoly/spinalcordtoolbox/jobs/116678146 for failing tests)
If so, let me now, I'll fix it quick (I just need to merge master on branch poq_issue_729 and do small changes)

just tried sct_testing to compare processing time with launcher. Results are not so good:

poq_issue_729 (82e0949273a9a7a1f6d7885902f891649bcfb69c): --> 308s
master (8c4796ad2d7d79346e79b0270950a9c3a00539aa): --> 180s

I've added a condition that setup sct_launcher variables only on the original call.
Linux: 227s
OSX: 490s

No path is modified when user 0 (super user) runs install_sct


Add a base64 encode/decode utility and maybe add to Calc as well

Options (for now, because I might want to encode/decode a string too):

-encode, -enc, -e = encode a file into a base 64 stream/file
-decode, -dec, -d = decode a base 64 stream into the original data
-file, -f FILE = (default) input from file
-- = input from stdin
-output, -out, -o FILE = the output file name (default stdout)

The Base64.InputStream and Base64.OutputStream need to be used.

The question on the options is how to match input names with output names.
Or else only allow one input at a time (either string or file).
Need a way to figure out that neither -file nor -string was specified (doesn't work that way right now)

Need results of #510 in order to test further.
Still missing any Calc support, and "--" as read from stdin.

Need an option to reformat an already encoded file to 76 char lines, or back to a single line...
In addition to the "--" option to read stdin.

Going to leave Calc support for later (and maybe another issue). Closing now.


DAX Error on Calculated Column

ON the relationships table of the PBIT/PBIX template
ToColumn = LOOKUPVALUE ( 'Columns Metadata'[Table], 'Columns Metadata'[ColumnID], 'Relationships'[ToColumnID] )
Should this not return Column Name and not the Table Name
Should be
FromColumn = LOOKUPVALUE ( 'Columns Metadata'[Column], 'Columns Metadata'[ColumnID], 'Relationships'[FromColumnID] )
Same applied to the FromColumn

Thanks for flagging this! I just updated the pbit file which should fix these issues.
Keep on contributing :) Much appreciated!
Marc


Eliminate &'static references and allow database updates without restarting

This can still be an inefficient batch operation, but let's at least put the data behind an Arc so we can swap it out, instead making a leaky static reference.
Don't need to do this all at once but probably:

[ ] Put database tables behinds Arc
[ ] Add a method(s) to replace tables with updated ones while maintaining integrity/validation
[ ] stop leaking memory during import
[ ] eliminate use of static references in download (even if those aren't leaky)
[ ] actually provide some way to update the data in serve (maybe hitting a secret URL)

Why do the records need to be static, now?
If they could be owned by their table, we could swap out the database+tables without leaking memory.


Trouble on Windows with Python 2.7.x 64 bit due to 'shape' being 'long' in this platform

Hi,
I am encountering some issues with cvxpy that are restricted to a single platform (python 2.7.x 64 bit on Windows, e.g. Winpython <IP_ADDRESS> 64 bit).
In this platform, differently from any other, the shape of an ndarray is given by a tuple of long integers, rather than 'int' integers. This should be just an implementation detail, but it creates issues with cvxpy.
Specifically, when you create expressions, the shapes easily start getting mixed int and long types. Eventually, cvxopt tries using long indexes in expressions and this seems to fail.
I think that some int casts are required in the cvxpy codebase.
As an example, try running the test suite of pydsm (you know this piece of code already... and I still need to thank you for the help and the code you prepared for a pull on my side!).

Sorry for taking so long to respond to this. I don't use windows and don't have a good setup for testing things on windows. Do you have any suggestions for ways to easily test on windows? I know about appveyor and have been meaning to set up appveyor tests for a while (@echo could you help with this?).
I hopefully fixed the issue in the latest patch to master, but I wasn't able to test the fix fully. Would you mind testing it on your machine?

Don't be sorry for taking long, as I need time to do to things on Windows as well...
In any case, I managed testing right now and the issue seems to be still there.
bbc3e96811ca9ac91905ca6cb53f8ec8cb43cf1d does not seem to be enough.
I'll try to see if I can create a self-contained test program (e.g. to add to a testsuite).

(Is this the same issue as https://github.com/cvxgrp/cvxpy/issues/127 ?)


User without superuser rights can remove installed items in trackable part, but cant install it.

Please verify that this bug has NOT been raised before.

[X] I checked and didn't find a similar issue

Describe the bug*
User without superuser rights can remove installed items in trackable part, but cant install it.
Steps to Reproduce
A1. Create user witout superuser checkbox
A2. Try to install item in trackable part (you cant)
B1. Create user witout superuser checkbox
B2. Try to remove item in trackable part (you do)
Expected behaviour
Staff must be able to do it, without superuser rights or even without stock add\remove rights (only change).
Deployment Method

[ ] Docker
[X] Bare metal

Version Information
Version Information:
InvenTree-Version: 0.13.3
Django Version: 3.2.23
Commit Hash: e81349e7
Commit Date: None
Commit Branch: stable
Database: postgresql
Debug-Mode: False
Deployed using Docker: False
Platform: Linux-5.4.0-156-generic-x86_64-with-glibc2.31
Installer: PKG
Target: ubuntu:20.04
Active plugins: [{'name': 'InvenTreeBarcode', 'slug': 'inventreebarcode', 'version': '2.0.0'}, {'name': 'InvenTreeCoreNotificationsPlugin', 'slug': 'inventreecorenotificationsplugin', 'version': '1.0.0'}, {'name': 'InvenTreeCurrencyExchange', 'slug': 'inventreecurrencyexchange', 'version': '1.0.0'}, {'name': 'InvenTreeLabel', 'slug': 'inventreelabel', 'version': '1.0.0'}, {'name': 'InvenTreeLabelSheet', 'slug': 'inventreelabelsheet', 'version': '1.0.0'}, {'name': 'DigiKeyPlugin', 'slug': 'digikeyplugin', 'version': '1.0.0'}, {'name': 'LCSCPlugin', 'slug': 'lcscplugin', 'version': '1.0.0'}, {'name': 'MouserPlugin', 'slug': 'mouserplugin', 'version': '1.0.0'}, {'name': 'TMEPlugin', 'slug': 'tmeplugin', 'version': '1.0.0'}]
Please verify if you can reproduce this bug on the demo site.

[ ] I can reproduce this bug on the demo site.

Relevant log output
No response

@alxrMironov I have not been able to reproduce this. Can you improve your example, perhaps use some parts / users from the demo server as an example?

@alxrMironov I have not been able to reproduce this. Can you improve your example, perhaps use some parts / users from the demo server as an example?

Can reproduce in demo.
Login as engineer and able to remove installed items from serialized.
But cant install another one.


Finalize conventions for recording LTS data releases

Motivation
We currently don't have a final system to tag and surface LTS data release in the release.json and the API respectively.
Definitions
Please see definition of data releases here.
Labels used by API in open_soma(census_version = ""):

"stable" is the latest LTS data release
"latest" is the current weekly release

User stories

I want to access the latest Census data release that contains datasets from as late as last week.
I want to access the latest LTS Census data release. And I would want to know which specific version it is so that in the future I can get use this exact version to get consistent results.
I want to access a specific LTS Census data release for consistency.
I want to understand if, why, and which LTS Census releases have been removed.

Product requirements
This system must:

Be backwards compatible with current implementation
Allow a future version of the API to identify all data releases available to open_soma.
Allow a future version of the API to identify all LTS data releases available to open_soma
The LTS release names surfaced to the user must be clear and indicate they are LTS, as well as their build dates.
open_soma should provide clear messaging of what version is being opened (already satisfied).
The default census_version = "stable" of open_soma must open the latest LTS data release.
census_version = "latest" of open_soma must open the current weekly release.
In the unlikely edge-case where an LTS is deleted:

The system must retire the data release and flag it as such.
They system must be able to record the reason for its removal.
The API must provide an error with clear messaging if a user tries to open it via open_soma

Proposed implementation, serves as guide for eng but final design is owned by @atolopko-czi
The release.json has the following conventions:

A dynamic alias "stable" that points to the latest LTS data release
A dynamic alias "latest" that points to the current weekly release
Each LTS data release must be named with the convention "LTS-[YY.MM]"
Each release must have a key is_LTS which must be True if the release is an LTS, False otherwise

The APIs must:

Allow to get the aliases of all LTS's
Provide clear information of the release being opened upon calling open_soma (already implemented)

Example release.json
{
"stable": "LTS-2022.10",
"latest": "2022-10-08",
"2022-10-08": {
    "release_build": "2022-10-08",
    "is_LTS": "False",
    "soma": {
        "uri": "s3://cellxgene-data-public/cell-census/2022-10-08/soma/",
        "s3_region": "us-west-2",
    },
    "h5ads": {
        "uri": "s3://cellxgene-data-public/cell-census/2022-10-08/h5ads/",
        "s3_region": "us-west-2",
    },
},
"LTS-22.10": {
    "release_build": "2022-10-01",
    "is_LTS": "True",
    "soma": {
        "uri": "s3://cellxgene-data-public/cell-census/2022-10-01/soma/",
        "s3_region": "us-west-2",
    },
    "h5ads": {
        "uri": "s3://cellxgene-data-public/cell-census/2022-10-01/h5ads/",
        "s3_region": "us-west-2",
    },
},
"LTS-22.05": {
    "release_build": "2022-05-01",
    "is_LTS": "True",
    "soma": {
        "uri": "s3://cellxgene-data-public/cell-census/2022-05-01/soma/",
        "s3_region": "us-west-2",
    },
    "h5ads": {
        "uri": "s3://cellxgene-data-public/cell-census/2022-05-01/h5ads/",
        "s3_region": "us-west-2",
    },
},
}

Another requirement to consider: "Ability to retract an LTS, and make this apparent to the user".
We should also decide if we're committed to the current release.json structure (for now) or whether we can evolve it when addressing this, since there may ways to improve the file structure. Spitballing:
{
  weekly: [
    { "2022-10-08": "2022-10-08" },
    { "2022-09-08": "2022-09-08" }
  ]
  stable: [
    { "LTS-2022-10": { build: "2022-10-08", retracted: null },
    { "LTS-2022-09": { build: "2022-09-08", retracted: "2022-10-07" }
  ]
  builds: {
    "2022-11-08": { "soma": ..., "h5ads": ... },
    "2022-10-08": { "soma": ..., "h5ads": ... },
    "2022-09-08": { "soma": ..., "h5ads": ... },
  }
}

This separates the "raw" builds from what they mean (weekly, stable, etc.). Still supports aliases. Supports LTS retraction. Supports explicit ordering of weekly/stable for display purposes (via use of json arrays).  Plenty of other ways to structure this, of course, so let's iterate in a meeting perhaps.

Note to self: above release.json is no longer that latest.


Memory/RAM solutions

We'll need to think about how and "where" to deal with RAM limitations, when dealing with e.g. many climate projections.
At the moment, in a 32GB machine, climr runs out of memory if all GCMs/periods/scenarios are requested (process_one_raster  runs out of memore when extracting the climate values)
Options are to (not mututally exclusive):

(partially) deal with these in climr
deal with these externally to climr (e.g. in the modules that will need to extract cliamte projections)
have climr warn the user about memory limitations (implemented in climr@out-of-memory-extract)

currently implementing options 2 and 3


Domo refresh developer token

Just for clarification, @Shamwayne waiting to approve until all 3 scripts have been updated with this authorization logic.


Can't run uv init due to uv init not being a subcommand

Hi, I installed uv with:
curl -LsSf https://astral.sh/uv/install.sh | sh

When I run uv init I get:
uv init example
error: unrecognized subcommand 'init'

Usage: uv [OPTIONS] <COMMAND>

For more information, try '--help'.

uv init is not a command given by uv
uv
Usage: uv [OPTIONS] <COMMAND>

Commands:
  pip      Resolve and install Python packages
  tool     Run and manage executable Python packages
  venv     Create a virtual environment
  cache    Manage the cache
  self     Manage the `uv` executable
  version  Display uv's version
  help     Print this message or the help of the given subcommand(s)

What gives? 🤔

Fixed! I had 2 versions of uv installed.

Thanks for following up!


AvatarGroup: Update addCollaborators icon from 'add' to 'add-person'

Before:

After:

Pull Request Template
Summary
Changing the icon used for addCollaborators prop on AvatarGroup
What changed?
At a high level, what changes does this PR introduce?
Changing the icon for add-collaborators.
Why?
Improved user comprehension

Huh…I'm surprised there aren't any snapshots that need to be updated?

@dangerismycat -
The visual snapshots don't have it:

And the it's set to false in the JSDOM snapshots
(https://github.com/pinterest/gestalt/blob/300667683bc15693751657e552705f1c018bb863/packages/gestalt/src/AvatarGroup.jsdom.test.js#L13)


Add HEALTHCHECKs for some services

[ ] server
[ ] dagent
[ ] spooler
[ ] gateway
[ ] search
[ ] webapp

#14 added the healthcheck to the Dockerfile, but its not doing anything at the moment.

With https://github.com/zokradonh/kopano-docker/pull/223 there are now some (still basic) checks


nitin feature get design settings

In this PR I have added a function to retrieve design settings based on prop names available in the object.
I have added UT for Icepak.

@nnetake Please could you add the missing tests? You can find them in the Files Changed section. Basicaly, there are 2 missing tests in two Try / excepts. If you can not test it because you can not reproduce it, try to find a design to reproduce it, if not you should add the line # pragam no cover to exclude this part. You can find some PyAEDT lines with this comment.


Update the instructions on the "key" page

This PR:

updates the instructions on the "key" page, including changing all the pictures
updates the FR info/warning banner on the "start" page

To see the changes, click the "generate key" button on this page: https://covid-alert-v6-key-c7c9ezaasm0.herokuapp.com/en/start/
(Note that when you switch between FR and EN on the "key" page itself, it will return you to the start page. This is intentional behaviour.)
Screenshots
It's hard to see because the screen is so long but here's how the top of it looks.

before
after

looks good to me, thanks @pcraig3

Removing the version bump as I can get a few more content changes in before releasing.


make the website name not have a hyphen when authenticating

Make it so when signing in, there Scratch Auth shows "Sign in to Snail IDE with Scratch" instead of "Sign in to Snail-IDE with Scratch"
I'm not sure if the hyphen is intentional, so sorry for making this if it is.

Some people say "Snail IDE" and some people say "Snail-IDE", but I will merge this since it's technically Snail IDE

nice


The recent update that addresses gk and gj

LazyVim Discussion Reference

I removed the "d" keys on hint and resetting keys did the trick for now but hoing youd have a fiix on it.

Please try out the latest version and try whether it works now.

The default config works smooth now, but the "gk" and "gj" isn't triggering the hardtime for some unknown reasons, was working sometimes like a week ago.

Can you provide your mapping of gj and gk? Or are you using the default config of LazyVim?

Can you provide your mapping of gj and gk? Or are you using the default config of LazyVim?

The default one from lazyvim, i believe this is it.
-- better up/down
map({ "n", "x" }, "j", "v:count == 0 ? 'gj' : 'j'", { expr = true, silent = true })
map({ "n", "x" }, "k", "v:count == 0 ? 'gk' : 'k'", { expr = true, silent = true })

This mapping works for me. Do you have any other config that might override this?

Thanks a lot.

d and key are doing its job, it's just not showing the whichkey menu, maybe its because of the delays set on which-key by default at lazyvim i am not sure tho, Thank you for fixing that! only solution to gk and gj is rebind it i think in keymaps for us lazyvim users. 🤖

Do you set the keymap before the plugin's setup? If yes, try to set the keymap after that.

@m4xshen, just to let you know, I'm facing the same problem.

@directormac please try out the latest version. Thanks.

Do you mean that gj and gk don't trigger hardtime? Do you also use LazyVim or use custom map for j and k?

I just use a custom map
vim.keymap.set("n", "k", "v:count == 0 ? 'gk' : 'k'", { noremap = true, expr = true, silent = true })
vim.keymap.set("n", "j", "v:count == 0 ? 'gj' : 'j'", { noremap = true, expr = true, silent = true })

I just use a custom map
vim.keymap.set("n", "k", "v:count == 0 ? 'gk' : 'k'", { noremap = true, expr = true, silent = true })
vim.keymap.set("n", "j", "v:count == 0 ? 'gj' : 'j'", { noremap = true, expr = true, silent = true })

After setting the keymap, the j and k don't trigger hardtime?

I just use a custom map
vim.keymap.set("n", "k", "v:count == 0 ? 'gk' : 'k'", { noremap = true, expr = true, silent = true })
vim.keymap.set("n", "j", "v:count == 0 ? 'gj' : 'j'", { noremap = true, expr = true, silent = true })

After setting the keymap, the j and k don't trigger hardtime?

Trying out hardline for the first time now. Hardtime fails to trigger for gj/gk.

Trying out hardline for the first time now. Hardtime fails to trigger with this keymap.

What Neovim version are you using? I can't reproduce the issue.

Also do you load the plugin first or set the keymap first?

Yup, that was the issue. Good shout.
This fixes it by making hardtime load after my keymaps:
return {
	{
		"m4xshen/hardtime.nvim",
+		event = "VeryLazy",
		opts = {}
	},
}

Thank you very much!

Also do you load the plugin first or set the keymap first?

Yup, that was the issue. Good shout.
This fixes it by making hardtime load after my keymaps:
return {
	{
		"m4xshen/hardtime.nvim",
+		event = "VeryLazy",
		opts = {}
	},
}

I tried doing this still j and k with gk and gj rebinding from lazyvim does not trigger hardtime,
thanks @m4xshen you fixed it finally i mised hardtime for the past few days.


typo: atheme -> a theme

Hi @nschonni - as PnP PowerShell documentation is auto-generated from the open-source code, this edit would be reset back within a month, unless we actually fix the issue in the originating file, which would be following file and line. Hopefully, you can resubmit the edit towards the actual code files, so that we can get this properly fixed - https://github.com/SharePoint/PnP-PowerShell/blob/dev/Commands/Admin/AddTenantTheme.cs#L18.
If you do not want to or have time to perform the edit towards correct file, please let us know, so that we can perform this manually.
Closing this PR as we do not accept edits on the PnP PowerShell documentation directly on the docs, as this documentation is autogenerated.


Adjusts to allow multiple Prescriptions

Some Adjustment to allow  creating multiples Prescriptions, and added some validation rules to limit ARV prescriptions

Coverage decreased (-0.2%) to 32.298% when pulling df7ab95a632dd4a6f919310f1ac9258b9c5c4d1a on Guimino:master into 4c9cfd7df625adc867efb3d4faa5070b49a71170 on esaude:master.

Coverage increased (+0.2%) to 32.672% when pulling 8540747ffba7ef19811e67614b0299e329c832f7 on Guimino:master into 4c9cfd7df625adc867efb3d4faa5070b49a71170 on esaude:master.

Coverage increased (+0.3%) to 32.801% when pulling db23daa597f1cb61df13ae0971433942a7cdea78 on Guimino:master into 4c9cfd7df625adc867efb3d4faa5070b49a71170 on esaude:master.

 1.0.1_36 Feature : Allow Refill mulptiple Prescriptions for Non ARV Prescriptions [8]

 1.0.1_37 Bug: Calculation of the amount of drugs to be picked up [?]

Coverage increased (+0.2%) to 32.7% when pulling 2e5217dced00969ac6dad4d25f8a1d559411ef44 on Guimino:master into 4c9cfd7df625adc867efb3d4faa5070b49a71170 on esaude:master.

 1.0.1_8 Feature: Display drug order status in pickup history [20]

 1.0.1_41 Bug: [Dispense in Pharmacy] If the user creates a new prescription the system cancel the previouse one [?]

 1.0.1_28 Feature: Cancel Dispensation


Start up error - Misconfigured Alarm Zone Bypass switch definition Zone Bypass ignoring

Getting an error in home bridge as follows - Misconfigured Alarm Zone Bypass switch definition Zone Bypass ignoring.
I am unable to further troubleshoot what is causing the error. Looking for some assistance please. Thanks
Logs:
10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Configuring Honeywell VISTA-20P for Homekit...
[10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Misconfigured Alarm Zone Bypass switch definition Zone Bypass ignoring.
[10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Partition configured: 1
[10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Zone accessories configured: 1
[10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Bypass accessories configured.
[10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Speed keys accessories configured.
[10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Starting connection to envisalink module at: <IP_ADDRESS>, port: 4025
Setup Payload:
X-HM://00247OG5HMHU3
[10/19/2022, 10:20:03 PM] Homebridge v1.5.0 (HAP v0.10.4) (Homebridge EC62) is running on port 51675.
[10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Successful TPI session established.
[10/19/2022, 10:20:03 PM] [Envisalink-Ademco] Checking for disconnected session every: 30 seconds.
Plugin Config:
{
"name": "Envisalink-Ademco",
"host": "<IP_ADDRESS>",
"port": 4025,
"deviceType": "20P",
"password": "user",
"pin": "4156",
"changePartition": false,
"openZoneTimeout": 30,
"heartbeatInterval": 30,
"commandTimeOut": 10,
"autoReconnect": true,
"sessionWatcher": true,
"chimeToggle": false,
"envisalinkFailureSuppress": false,
"maintenanceMode": false,
"partitions": [
{
"name": "House",
"partitionPin": "4156"
}
],
"zones": [
{
"name": "Mud Door",
"partition": "1",
"zoneNumber": "4",
"sensorType": "door",
"bypassEnabled": false
}
],
"bypass": [
{
"enabledbyPass": false
}
],
"speedKeys": [
{
"name": "Test",
"speedcommand": "A"
}
],
"platform": "Envisalink-Ademco"
Environment:
Docker 20.10.3-1308
Plugin Version: V 2.0.2
Homebridge Version: V 1.5.0
Node.js Version: 12.22.7-2029 (Installed in Synology DSM)
NPM Version:
Operating System: Synology Nas DSM 7.1.1-42962 Update 1
Docker reports OS: Ubuntu Focal Fossa (20.04.4 LTS)

Hi,
The error message can be ignored since bypass switch is disabled. It is caused by a logic flow that is no longer needed and will be removed in the next release. If you want to remove it prior to the next release; delete the following section in the configuration:
"bypass": [
{
"enabledbyPass": false
}

Update version 2.0.3 corrected the "Misconfigured Alarm Zone Bypass" message  if Bypass switch is not configured.


French translation

Hello,
I propose you the french translation.
Thank's for this plugin.
BR

Ooop's :) I commited the wrong file.. It's fixed.


Splitted value and unit of the result, made response delay configurable

Splitted value and unit of the result, made response delay configurable, typo

It will need rebase.

#38 or whatever comes next seems to close this.


HSEARCH-2520 Support analyzer definitions with the Elasticsearch MERGE strategy

This PR is based on #1270 (HSEARCH-2519), which should be merged first. => Done
Relevant JIRA ticket: https://hibernate.atlassian.net/browse/HSEARCH-2520
Whether we should merge that during a CR still has to be determined. It's probably too much change, but the PR can still be merged later (5.6.0.CR2? 5.7.0 ? 5.6.1? 5.8?)
The merge strategy now closes indexes when updating analyzer definitions. This is mandatory, we can't update (or even add) analyzer definitions without closing indexes.
This means in particular that, in environments where multiple clients access a same Elasticsearch index, users will have to set up some sort of synchronization mechanisms to avoid other clients to access the index while Hibernate Search is updating the analyzer definitions.
Handling this synchronization ourselves seems like a bad idea for multiple reasons. This would basically boil down to making the JGroup clustering mandatory when using multiple Hibernate Search instances even with Elasticsearch, which defeats one of the purposes of using Elasticsearch in the first place.
I'd say it's not that bad, because altering analyzer definitions is no small change anyway: in most cases it would require reindexing.
The only thing that bothers me is that we have to stop the index even when simply adding an analyzer, which is a shame. Maybe I should submit a feature request to the Elasticsearch team? I'd have to test ES 5 first, though.
EDIT: @gsmet, I looked into throwing specially-crafted exceptions when we have an error because of a closed index. It appears we can't do it with bulk requests, because those may target multiple indexes, which means even if one operation fails because of a closed index, this wouldn't always be true for all operations. So we can't decently make the whole BulkRequestFailedException state that the issue was with a closed index.
Knowing that we can only do it for single requests, I wonder if it's worth it? Mind that Elasticsearch error message are really obvious ("type": "index_closed_exception", "reason": "closed").

Jenkins, test this please

jenkins retest this please

merged!


Change scope of docs anchor preventDefault to include modals.

Fixes #20093 (the first point, the second being a duplicate of #19270). Needs sanity check to ensure I'm not breaking any other behavior.

I'm going to try and be bold here, and merge my first pull request. Fingers crossed this doesn't blow up in my face.


[SMALLFIX]Use static imports for standard test utilities in MostAvailableFirstPolicyTest.java

Change
import org.junit.Assert;
to
import static org.junit.Assert.assertEquals;
AND
Update all
Assert.{{ method }}*
to
{{ method }}*

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/Alluxio-Pull-Request-Builder/3542/
Test PASSed.

alluxio-bot, check this please

Automated checks report:

Commits associated with Github account: PASS

PR title follows the conventions: FAIL

The title of the PR does not pass all the checks. Please fix the following issues:

Title is too long (82 characters). Must be at most 72 characters.

Some checks failed. Please fix the reported issues and reply 'alluxio-bot, check this please' to re-run checks.

alluxio-bot, check this please

alluxio-bot, check this please.

alluxio-bot, merge this please


Add Series.get and fix getitem.

Codecov Report

Merging #1153 into master will decrease coverage by 2.52%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1153      +/-   ##
==========================================
- Coverage   95.16%   92.64%   -2.53%
==========================================
  Files          35       35
  Lines        7080     7081       +1
==========================================
- Hits         6738     6560     -178
- Misses        342      521     +179

Impacted Files
Coverage Δ

databricks/koalas/missing/series.py
100% <ø> (ø)
:arrow_up:

databricks/koalas/frame.py
93.62% <ø> (-3.26%)
:arrow_down:

databricks/koalas/series.py
96.36% <100%> (-0.13%)
:arrow_down:

databricks/koalas/generic.py
96.26% <100%> (+0.07%)
:arrow_up:

databricks/koalas/usage_logging/__init__.py
24.32% <0%> (-72.98%)
:arrow_down:

databricks/koalas/usage_logging/usage_logger.py
50% <0%> (-50%)
:arrow_down:

databricks/koalas/__init__.py
78.72% <0%> (-6.39%)
:arrow_down:

databricks/conftest.py
92% <0%> (-6%)
:arrow_down:

databricks/koalas/groupby.py
89.62% <0%> (-2%)
:arrow_down:

... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 899b346...da56b76. Read the comment docs.

Thanks. merging.


cant resolve path nor types on import

Hi :wave:
I was really looking forward for a nock like approach to graphql mocking. I followed your readme, but failed miserably to even import NockGraphQL in my monorepo x lerna setup sporting typescript 4.3.5.
'NockGraphQL' is declared but its value is never read. ts (6133)
Unable to resolve path to module 'nock-graphql'. eslint (import/no-unresolved)
Cannot find module 'nock-graphql' or its corresponding type declarations. ts(2307)

The whole Apollo migration to v3 already causes a lot of trouble. And for whatever reason (solution still pending) you have to include "lib": ["dom"] in your tsconfig.ts to fix a lot of the typing issues describe in that thread.
I don't know if that's much to go on, but maybe you have an idea?

Hi,
Thanks for letting me know of this issue. It is now fixed in the latest release (v1.1.6)


Updating package

Hi,
What would be the best way to update the package using composer when you make a new release?
I have "ohmybrew/laravel-shopify": "^2.0" in my composer.json, but when I run composer update ohmybrew/laravel-shopify, nothing happens.
Also, will this update your ohmybrew/basic-shopify-api package at the same time?
Thanks

It may be worth checking out composer versions here.  Once you determine the correct version and update your composer file, run composer update ohmybrew/laravel-shopify --with-dependencies
Hope this helps.

Adding --with-dependencies did the trick. The update went through
Package operations: 0 installs, 5 updates, 0 removals
  - Updating doctrine/common (v2.7.3 => v2.8.1): Loading from cache
  - Updating doctrine/dbal (v2.5.13 => v2.7.1): Loading from cache
  - Updating guzzlehttp/guzzle (6.3.0 => 6.3.3): Loading from cache
  - Updating ohmybrew/basic-shopify-api (v1.0.1 => v3.0.1): Loading from cache
  - Updating ohmybrew/laravel-shopify (v2.2.0 => v2.3.1): Loading from cache

The thing is after updating, when trying to view the app in my store I get:
Illuminate \ Database \ QueryException (HY000)
SQLSTATE[HY000]: General error: PDO::ATTR_STATEMENT_CLASS requires format array(classname, array(ctor_args)); the classname must be a string specifying an existing class (SQL: select * from `shops` where (`shopify_domain` = my-dev-store.myshopify.com) limit 1)

Googling this seems to suggest people downgraded to 2.6 of Doctrine DBAL. Others say the PHP version caused the issue. So I am not sure... What's the version of Laravel and PHP you're using?

PHP Version 7.0.12
Laravel 5.5.34

Googling seems to say the DBAL version is incompatible with that PHP version... hmm, can you try forcing doctrine/dbal to be ^2.5.0 in your composer.json?

I updated my PHP version to 7.1.13, updated the packages and now it works.
Maybe we should add a note in the readme saying that >= 2.3.0 version requires PHP >= 7.1.x?

Yeah, it's more the fault of something specific in the doctrine DBAL. I may
just fix it's version to be lower. I'll think on this. But for now I'll
close and thank you for testing this.
On Thu, May 31, 2018, 10:30 AM Marc-Antoine Brodeur, <
<EMAIL_ADDRESS>wrote:

I updated my PHP version to 7.1.13, updated the packages and now it works.
Maybe we should add a note in the readme saying that >= 2.3.0 version
requires PHP >= 7.1.x?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ohmybrew/laravel-shopify/issues/31#issuecomment-393549419,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACTuOsUEPtYGqsSN5G5-DLElraFl8C76ks5t3_55gaJpZM4UT9rq
.

Based on research, I may just bump the package to 7.1 minimum. 7.0 is rather old at this point and forcing an old version of DBAL may not be ideal.

Resolved in master, will be in next version push, thanks.


Add ya2yaml

Hi myronmarston, I added ya2yaml to make the dump yaml file more readable.

ok, i see=)


Barnhark/add to xarray

Addresses #88
This adds dependency on xarray and dask. Hopefully I put it into the CI requirements correctly.

Looks like these errors are all about AppVeyor not being able to import dask. Will attempt to fix.


[TW#15700] WPA2 enterprise example not working with eduroam

I got myself this ESP32 development board
and installed the toolchain as described here
When I now use the wpa2_enterprise example, configured (make menuconfig) with WiFI SSID "eduroam", EAP method 1, my email as EAP1 ID, EAP user name<EMAIL_ADDRESS>and password as described at my home institution, this compiles and uploads:

WARNING: Toolchain version is not supported: 1.22.0-61-gab8375a
Expected to see version: 1.22.0-73-ge28a011
Please check ESP-IDF setup instructions and update the toolchain, or proceed at your own risk.
WARNING: Toolchain version is not supported: 1.22.0-61-gab8375a
Expected to see version: 1.22.0-73-ge28a011
Please check ESP-IDF setup instructions and update the toolchain, or proceed at your own risk.
Flashing binaries to serial port /dev/tty.SLAB_USBtoUART (app at offset 0x10000)...
esptool.py v2.1
Traceback (most recent call last):
File "/Users/maurera/esp/esp-idf/components/esptool_py/esptool/esptool.py", line 2524, in
_main()
File "/Users/maurera/esp/esp-idf/components/esptool_py/esptool/esptool.py", line 2517, in _main
main()
File "/Users/maurera/esp/esp-idf/components/esptool_py/esptool/esptool.py", line 2246, in main
esp = chip_class(args.port, initial_baud)
File "/Users/maurera/esp/esp-idf/components/esptool_py/esptool/esptool.py", line 177, in init
self._port = serial.serial_for_url(port)
File "/Library/Python/2.7/site-packages/serial/init.py", line 78, in serial_for_url
instance.open()
File "/Library/Python/2.7/site-packages/serial/serialposix.py", line 289, in open
self.fd = os.open(self.portstr, os.O_RDWR|os.O_NOCTTY|os.O_NONBLOCK)
OSError: [Errno 16] Resource busy: '/dev/tty.SLAB_USBtoUART'
make: *** [flash] Error 1
mbple-3:wpa2_enterprise maurera$ make flash
WARNING: Toolchain version is not supported: 1.22.0-61-gab8375a
Expected to see version: 1.22.0-73-ge28a011
Please check ESP-IDF setup instructions and update the toolchain, or proceed at your own risk.
WARNING: Toolchain version is not supported: 1.22.0-61-gab8375a
Expected to see version: 1.22.0-73-ge28a011
Please check ESP-IDF setup instructions and update the toolchain, or proceed at your own risk.
Flashing binaries to serial port /dev/tty.SLAB_USBtoUART (app at offset 0x10000)...
esptool.py v2.1
Connecting......
Chip is ESP32D0WDQ6 (revision 1)
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Auto-detected Flash size: 4MB
Flash params set to 0x0220
Compressed 17776 bytes to 10472...
Wrote 17776 bytes (10472 compressed) at 0x00001000 in 0.9 seconds (effective 153.1 kbit/s)...
Hash of data verified.
Compressed 509920 bytes to 325366...
Wrote 509920 bytes (325366 compressed) at 0x00010000 in 28.8 seconds (effective 141.7 kbit/s)...
Hash of data verified.
Compressed 3072 bytes to 82...
Wrote 3072 bytes (82 compressed) at 0x00008000 in 0.0 seconds (effective 2051.7 kbit/s)...
Hash of data verified.
Leaving...
Hard resetting...

but doesn't connect to eduroam:

rst:0x10 (RTCWDT_RTC_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0010,len:4
load:0x3fff0014,len:5492
load:0x40078000,len:0
load:0x40078000,len:12180
entry 0x40078f54
W (33) rtc_clk: Possibly invalid CONFIG_ESP32_XTAL_FREQ setting (40MHz). Detected 40 MHz.
I (57) boot: ESP-IDF v3.0-dev-753-g845c3fb-dirty 2nd stage bootloader
I (58) boot: compile time 11:05:26
I (62) boot: Enabling RNG early entropy source...
I (78) boot: SPI Speed      : 40MHz
I (91) boot: SPI Mode       : DIO
I (103) boot: SPI Flash Size : 4MB
I (116) boot: Partition Table:
I (127) boot: ## Label            Usage          Type ST Offset   Length
I (150) boot:  0 nvs              WiFi data        01 02 00009000 00006000
I (173) boot:  1 phy_init         RF data          01 01 0000f000 00001000
I (196) boot:  2 factory          factory app      00 00 00010000 00100000
I (220) boot: End of partition table
I (233) esp_image: segment 0: paddr=0x00010020 vaddr=0x3f400020 size=0x13654 ( 79444) map
I (345) esp_image: segment 1: paddr=0x0002367c vaddr=0x3ffb0000 size=0x030c0 ( 12480) load
I (361) esp_image: segment 2: paddr=0x00026744 vaddr=0x40080000 size=0x00400 (  1024) load
I (367) esp_image: segment 3: paddr=0x00026b4c vaddr=0x40080400 size=0x094c4 ( 38084) load
I (442) esp_image: segment 4: paddr=0x00030018 vaddr=0x400d0018 size=0x5676c (354156) map
I (820) esp_image: segment 5: paddr=0x0008678c vaddr=0x400898c4 size=0x06020 ( 24608) load
I (852) esp_image: segment 6: paddr=0x0008c7b4 vaddr=0x400c0000 size=0x00000 (     0) load
I (881) boot: Loaded app from partition at offset 0x10000
I (881) boot: Disabling RNG early entropy source...
I (884) cpu_start: Pro cpu up.
I (894) cpu_start: Starting app cpu, entry point is 0x40080ed0
I (0) cpu_start: App cpu up.
I (927) heap_init: Initializing. RAM available for dynamic allocation:
I (948) heap_init: At 3FFAE6E0 len 00001920 (6 KiB): DRAM
I (966) heap_init: At 3FFB8958 len 000276A8 (157 KiB): DRAM
I (986) heap_init: At 3FFE0440 len 00003BC0 (14 KiB): D/IRAM
I (1005) heap_init: At 3FFE4350 len 0001BCB0 (111 KiB): D/IRAM
I (1025) heap_init: At 4008F8E4 len 0001071C (65 KiB): IRAM
I (1044) cpu_start: Pro cpu start user code
I (1105) cpu_start: Starting scheduler on PRO CPU.
I (197) cpu_start: Starting scheduler on APP CPU.
I (207) wifi: wifi firmware version: 731ced4
I (207) wifi: config NVS flash: enabled
I (207) wifi: config nano formating: disabled
I (207) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (217) system_api: Base MAC address is not set, read default base MAC address from BLK0 of EFUSE
I (247) wifi: Init dynamic tx buffer num: 32
I (247) wifi: Init data frame dynamic rx buffer num: 32
I (247) wifi: Init management frame dynamic rx buffer num: 32
I (247) wifi: wifi driver task: 3ffc5288, prio:23, stack:4096
I (257) wifi: Init static rx buffer num: 10
I (257) wifi: Init dynamic rx buffer num: 32
I (257) wifi: Init rx ampdu len mblock:7
I (267) wifi: Init lldesc rx ampdu entry mblock:4
I (267) wifi: wifi power manager task: 0x3ffca6f4 prio: 21 stack: 2560
I (277) example: Setting WiFi configuration SSID eduroam...
I (287) wpa: WPA2 ENTERPRISE VERSION: [v2.0] enable
I (317) phy: phy_version: 359.0, e79c19d, Aug 31 2017, 17:06:07, 0, 0
I (317) wifi: mode : sta (24:0a:c4:0e:0d:b0)
I (1037) wifi: n:6 0, o:1 0, ap:255 255, sta:6 0, prof:1
I (2017) wifi: state: init -> auth (b0)
I (2017) wifi: state: auth -> assoc (0)
I (2027) wifi: state: assoc -> run (10)
I (2027) wpa: wpa2_task prio:2, stack:6656
I (4037) wpa: >>>>>wpa2 FIALED
I (4047) wpa: wpa2 task delete
I (4317) example: ~~~~~~~~~~~
I (4317) example: IP:<IP_ADDRESS>
I (4317) example: MASK:<IP_ADDRESS>
I (4317) example: GW:<IP_ADDRESS>
I (4317) example: ~~~~~~~~~~~
I (5027) wifi: pm start, type:0
I (6317) example: ~~~~~~~~~~~
I (6317) example: IP:<IP_ADDRESS>
I (6317) example: MASK:<IP_ADDRESS>
I (6317) example: GW:<IP_ADDRESS>
I (6317) example: ~~~~~~~~~~~
I (8317) example: ~~~~~~~~~~~
I (8317) example: IP:<IP_ADDRESS>
I (8317) example: MASK:<IP_ADDRESS>
I (8317) example: GW:<IP_ADDRESS>
I (8317) example: ~~~~~~~~~~~
I (10317) example: ~~~~~~~~~~~
I (10317) example: IP:<IP_ADDRESS>
I (10317) example: MASK:<IP_ADDRESS>
I (10317) example: GW:<IP_ADDRESS>
I (10317) example: ~~~~~~~~~~~
I (12317) example: ~~~~~~~~~~~
I (12317) example: IP:<IP_ADDRESS>
I (12317) example: MASK:<IP_ADDRESS>
I (12317) example: GW:<IP_ADDRESS>
I (12317) example: ~~~~~~~~~~~
I (14317) example: ~~~~~~~~~~~
I (14317) example: IP:<IP_ADDRESS>
I (14317) example: MASK:<IP_ADDRESS>
I (14317) example: GW:<IP_ADDRESS>
I (14317) example: ~~~~~~~~~~~
I (16317) example: ~~~~~~~~~~~
I (16317) example: IP:<IP_ADDRESS>
I (16317) example: MASK:<IP_ADDRESS>
I (16317) example: GW:<IP_ADDRESS>
I (16317) example: ~~~~~~~~~~~
I (18317) example: ~~~~~~~~~~~
I (18317) example: IP:<IP_ADDRESS>
I (18317) example: MASK:<IP_ADDRESS>
I (18317) example: GW:<IP_ADDRESS>
I (18317) example: ~~~~~~~~~~~
I (20317) example: ~~~~~~~~~~~
I (20317) example: IP:<IP_ADDRESS>
I (20317) example: MASK:<IP_ADDRESS>
I (20317) example: GW:<IP_ADDRESS>
I (20317) example: ~~~~~~~~~~~
I (22317) example: ~~~~~~~~~~~
I (22317) example: IP:<IP_ADDRESS>
I (22317) example: MASK:<IP_ADDRESS>
I (22317) example: GW:<IP_ADDRESS>
I (22317) example: ~~~~~~~~~~~
I (24317) example: ~~~~~~~~~~~
I (24317) example: IP:<IP_ADDRESS>
I (24317) example: MASK:<IP_ADDRESS>
I (24317) example: GW:<IP_ADDRESS>
I (24317) example: ~~~~~~~~~~~
I (26317) example: ~~~~~~~~~~~
I (26317) example: IP:<IP_ADDRESS>
I (26317) example: MASK:<IP_ADDRESS>
I (26317) example: GW:<IP_ADDRESS>
I (26317) example: ~~~~~~~~~~~
I (28317) example: ~~~~~~~~~~~
I (28317) example: IP:<IP_ADDRESS>
I (28317) example: MASK:<IP_ADDRESS>
I (28317) example: GW:<IP_ADDRESS>
I (28317) example: ~~~~~~~~~~~
I (30317) example: ~~~~~~~~~~~
I (30317) example: IP:<IP_ADDRESS>
I (30317) example: MASK:<IP_ADDRESS>
I (30317) example: GW:<IP_ADDRESS>
I (30317) example: ~~~~~~~~~~~
I (32027) wifi: state: run -> init (2)
I (32027) wifi: pm stop, total sleep time: 0/27001132
I (32027) wifi: n:6 0, o:6 0, ap:255 255, sta:6 0, prof:1
I (32157) wifi: n:6 0, o:6 0, ap:255 255, sta:6 0, prof:1
I (32157) wifi: state: init -> auth (b0)
I (32157) wifi: state: auth -> assoc (0)
I (32167) wifi: state: assoc -> run (10)
I (32167) wpa: wpa2_task prio:2, stack:6656
I (32317) example: ~~~~~~~~~~~
I (32317) example: IP:<IP_ADDRESS>
I (32317) example: MASK:<IP_ADDRESS>
I (32317) example: GW:<IP_ADDRESS>
I (32317) example: ~~~~~~~~~~~
I (34237) wpa: >>>>>wpa2 FIALED
I (34247) wpa: wpa2 task delete
I (34317) example: ~~~~~~~~~~~
I (34317) example: IP:<IP_ADDRESS>
I (34317) example: MASK:<IP_ADDRESS>
I (34317) example: GW:<IP_ADDRESS>
I (34317) example: ~~~~~~~~~~~
I (35167) wifi: pm start, type:0
I (36317) example: ~~~~~~~~~~~
I (36317) example: IP:<IP_ADDRESS>
I (36317) example: MASK:<IP_ADDRESS>
I (36317) example: GW:<IP_ADDRESS>
I (36317) example: ~~~~~~~~~~~
I (38317) example: ~~~~~~~~~~~
I (38317) example: IP:<IP_ADDRESS>
I (38317) example: MASK:<IP_ADDRESS>
I (38317) example: GW:<IP_ADDRESS>
I (38317) example: ~~~~~~~~~~~

Adding the Telekom root certificate as wpa2_ca.pem doesn't seem to change anything. Is there anything I could do, and to get more details where/how it failed?

Hi @maur3ra Sorry for later reply, have you tried with our latest IDF? What's more, have you udpate the router server's certificate in the example's main folder? There is another customer also met the problem, we suggest them update their router server's certificate and problem solved. There are two certificates wpa_server.key and wpa_server.pem, you need copy them to the router server and replace the original, after change them, rename the name to server.key and server.pem

Hi @XinDeng11 , thanks for the helpful reply! You suggest changing the certificate on the router? Or Where exactly would the router server sit within the eduroam infrastructure (world-wide between schools and universities)? As far as I know, the authentication is done by the server at the "home organization" (my university), nevertheless at which partner site and in which building you are.  And I probably don't have access to the router, right?
On eduroam, with the iPhone I just connect using username and password, on Android devices you also have to add the root certificate. Do you suggest the opposite (changing it on the server rather than getting the server certificate), or did I misunderstand you?

@maur3ra - This is probably too late to ask for, but if you still face the issue with latest release, we will actively look into it. Thank you and apologies for the delay!

Closing this issue because of lack of updates. Feel free to re-open if the issue is observed again.


Restore add-org.spec to passing

Overview
add-org.spec.js has been flaky, causing unrelated Pull Requests to faill
Action Items

[ ] Figure out why 3 (or so) of the tests are failing.
[ ] Fix them or disable them or write new tests, as needed.

@bruceplai please update the following
•	Progress: How much progress have you made on the issue?
•	Blocks: Are you facing any blockers?
•	Availability: How much time do you have to work on it this week?
•	ETA: When do you think it will be completed?