Read basic information (CON-290)

I'm trying to read the basic information of the device.
with mfg_tool I created the partition with all the data (manufacturer, model, serial...)
if i try to read this data with chip-tool i read it correctly
./chip-tool any read-by-id 0x28 0x1 0x11 0
but if I try to read them from code they return empty
cluster_t *basic_information_cluster = cluster::get(endpoint, BasicInformation::Id);
attribute_t *basic_information_attribute = attribute::get(basic_information_cluster, BasicInformation::Attributes::VendorName::Id);
esp_matter_attr_val_t basic_information_val = esp_matter_invalid(NULL);
attribute::get_val(basic_information_attribute, &basic_information_val);
if (basic_information_val.type == ESP_MATTER_VAL_TYPE_CHAR_STRING){
ESP_LOGI(TAG, "********** Endpoint 0x%04X's Cluster 0x%04X's Attribute 0x%04X is %.*s **********", 0,
BasicInformation::Id, BasicInformation::Attributes::VendorName::Id, basic_information_val.val.a.s, basic_information_val.val.a.b);
}
its the same if i try to read with matter esp console

matter esp attribute get 0x0 0x28 0x1
I (432668) esp_matter_attribute: ********** Endpoint 0x0000's Cluster 0x0028's Attribute 0x0001 is  **********
I (432670) esp_matter_attribute: ********** Endpoint 0x0000's Cluster 0x0028's Attribute 0x0001 is  **********
Done

I added some Kconfig and put this in CMakelists.txt. I am not finding mfg_tool useful until production time. You probably did not copy the mfg_partition onto the device.
message(STATUS "Vendor ID: " ${CONFIG_DEVICE_VENDOR_ID})
message(STATUS "Vendor Name: " ${CONFIG_DEVICE_VENDOR_NAME})
message(STATUS "Product ID: " ${CONFIG_DEVICE_PRODUCT_ID})
message(STATUS "Product Name: " ${CONFIG_DEVICE_PRODUCT_NAME})
message(STATUS "SW Version String: " ${CHIP_DEVICE_CONFIG_DEVICE_SOFTWARE_VERSION_STRING})
idf_build_set_property(COMPILE_OPTIONS "-DCHIP_DEVICE_CONFIG_DEVICE_VENDOR_ID=${CONFIG_DEVICE_VENDOR_ID}" APPEND)
idf_build_set_property(COMPILE_OPTIONS -DCHIP_DEVICE_CONFIG_DEVICE_VENDOR_NAME="${CONFIG_DEVICE_VENDOR_NAME}" APPEND)
idf_build_set_property(COMPILE_OPTIONS "-DCHIP_DEVICE_CONFIG_DEVICE_PRODUCT_ID=${CONFIG_DEVICE_PRODUCT_ID}" APPEND)
idf_build_set_property(COMPILE_OPTIONS -DCHIP_DEVICE_CONFIG_DEVICE_PRODUCT_NAME="${CONFIG_DEVICE_PRODUCT_NAME}" APPEND)
if(NOT ${CHIP_DEVICE_CONFIG_DEVICE_SOFTWARE_VERSION_STRING} STREQUAL "")
    idf_build_set_property(COMPILE_OPTIONS "-DCHIP_DEVICE_CONFIG_DEVICE_SOFTWARE_VERSION_STRING=\"${CHIP_DEVICE_CONFIG_DEVICE_SOFTWARE_VERSION_STRING}\"" APPEND)
endif()

Hi @jonsmirl , I created the partition and wrote it on the device and in fact with chip-tool I can correctly read the vendor name, product name,...
but i can't read them from the code
cluster_t *basic_information_cluster = cluster::get(endpoint, BasicInformation::Id);
attribute_t *basic_information_attribute = attribute::get(basic_information_cluster, BasicInformation::Attributes::VendorName::Id);
esp_matter_attr_val_t basic_information_val = esp_matter_invalid(NULL);
attribute::get_val(basic_information_attribute, &basic_information_val);
if (basic_information_val.type == ESP_MATTER_VAL_TYPE_CHAR_STRING){
ESP_LOGI(TAG, "********** Endpoint 0x%04X's Cluster 0x%04X's Attribute 0x%04X is %.*s **********", 0,
BasicInformation::Id, BasicInformation::Attributes::VendorName::Id, basic_information_val.val.a.s, basic_information_val.val.a.b);
}
its the same if i try to read with matter esp console
matter esp attribute get 0x0 0x28 0x1
I (432668) esp_matter_attribute: ********** Endpoint 0x0000's Cluster 0x0028's Attribute 0x0001 is **********
I (432670) esp_matter_attribute: ********** Endpoint 0x0000's Cluster 0x0028's Attribute 0x0001 is **********
Done

@stefano055415 Some of the generic attributes are internally managed by the Matter stack and the callbacks for read/write are not given to the application. The basic information cluster attributes are managed that way.
Also, if you see, the esp-matter data model does not allocate memory for such attributes. See this.
If you want to locally read such attributes, you can do so from the factory partition. Please see how it is read from the Matter stack here

@dhrishi
Ok perfect, thanks

Hi @jonsmirl , i tried your suggestion to add the lines to the CMakeLists.txt but i while building i get this error: "I have no idea what this is." After removing the lines from CMakeLists.txt it is working.
Added Lines:
message(STATUS "Vendor Name: " ${CONFIG_DEVICE_VENDOR_NAME}) message(STATUS "Product Name: " ${CONFIG_DEVICE_PRODUCT_NAME}) message(STATUS "HW Version String: " ${CONFIG_DEVICE_HARDWARE_VERSION_STRING}) idf_build_set_property(COMPILE_OPTIONS -DCHIP_DEVICE_CONFIG_DEVICE_VENDOR_NAME="${CONFIG_DEVICE_VENDOR_NAME}" APPEND) idf_build_set_property(COMPILE_OPTIONS -DCHIP_DEVICE_CONFIG_DEVICE_PRODUCT_NAME="${CONFIG_DEVICE_PRODUCT_NAME}" APPEND) if(NOT ${CHIP_DEVICE_CONFIG_DEVICE_HARDWARE_VERSION_STRING} STREQUAL "")     idf_build_set_property(COMPILE_OPTIONS "-DCHIP_DEVICE_CONFIG_DEVICE_HARDWARE_VERSION_STRING=\"${CONFIG_DEVICE_HARDWARE_VERSION_STRING}\"" APPEND) endif()
The same error occurs when removing the last 4 lines which i modified from your suggestion for adding the hardware version string
Do you have any idea what could cause this?
Where exactly do i have to place these lines?


Fix boolean conversion in json-xml-json conversion

When setting context variables to boolean values like true / false, they get converted to “True” and “False”, which evaluates differently in node checks.

[x] Add boolean value to attribute type
[x] Add documentation
[x] Add tests

All done, closing.


The "More" menu was redesigned, so adding Add Muted Word no longer works, and some of the options for hiding items no longer make sense

It now looks like this:


"Download audio" isn't working with local files

https://github.com/killergerbah/asbplayer/commit/4719376409edc0eada0bea3a5ee441dca3803a3c


Limit maximum velocity

It should be possible to set a cap on the maximum linear and angular velocities of bodies. This could be used to improve stability, prevent velocities from ever becoming infinite (although of course that shouldn't happen anyway), and to allow users to limit things like the maximum movement speed of their characters.
This should be configurable at the entity level using components:
#[derive(Component)]
pub struct MaxLinearVelocity(pub Vector);

#[cfg(feature = "2d")]
#[derive(Component)]
pub struct MaxAngularVelocity(pub Scalar);

#[cfg(feature = "3d")]
#[derive(Component)]
pub struct MaxAngularVelocity(pub Vector);

And ideally also at a global level using some resource. SolverConfig could be an option, but we could also create a new, more discoverable resource that also has configuration for things like default damping and material properties (see #119).
Open Questions

What resource do we put the default configuration in?
Do we only apply the clamping before position and velocity integration, or everywhere that uses velocities? (maybe check what Box2D does here for reference)

I'm no physics expert so take my comment with a grain of salt because it may not be super good
but for the first question, I think it would be a good idea to just have it as part of the rigidbody component like LinearVelocity is, and for the second question I think it would make sense to have it everywhere that has a velocity since someone may, say, want a box to fall at a maximum of 1 meter for second for gameplay purposes


How to merge cell based on sheet?

The options only take effect on first sheet only. When i move into the loop, the excel became corrupted. How to achieve this?
$writer = WriterFactory::createFromFile($path);
$writer->getOptions()->mergeCells(0, 1, 3, 1);
$writer->openToFile($path);

for loop {
   // add row
   // $writer->addNewSheetAndMakeItCurrent();
}

$writer->close();

I'm not sure what the issue is, but have you tried playing around with the 5th parameter of the mergeCells method?

Ah.. there is 5th params.. Need to drill down the code instead of documentation 😅


How to use PCMD command in Piloting

I wish to use PMCD command to move the Bebop2 to a relative point from current position.
I have tried:

drone.send_data('ardrone3.Piloting.PCMD( 1, 1, 0, 0 , 0 )')
but I got error:
Bad command !'Unknown command PCMD( 1, 1, 0, 0 , 0 ) in class Piloting of project ardrone3'
1

help(Bybop_Device.Device.send_data) should give you the answer.


New issue report form

SDK-1614 -- Github Issue Template for SDK repos
Introduces the new issue report form - references https://github.com/BranchMetrics/web-branch-deep-linking-attribution/issues/new/choose

I'll add it, if it doesn't make sense we can remove it later.


X Windows freezes on launch since upgrading to 17.03

Issue description
Ever since I upgraded from 16.09 to 17.03, X Windows freezes immediately after login. I've tried disabling gdm and using the NixOS default display manager (I believe SLiM?) as well as disabling gnome and xmonad. The problem persists unless I disable X server entirely, even with just the default display manager and xterm.
Steps to reproduce
My configuration.nix: https://github.com/Sophia-Gold/configuration.nix/blob/master/configuration.nix)
Journalctl dumps:

With gdm + gnome3 + xmonad: https://gist.github.com/Sophia-Gold/31d56817b0300cebbc2fbcd6de4e64fa
With gdm disabled: https://gist.github.com/Sophia-Gold/66d46601d8c99ab0228896dd4be361ac
With gdm and gnome3 disabled: https://gist.github.com/Sophia-Gold/b9f7246286882b03c8eadcfdaf0e15f9
Default display manager and xterm: https://gist.github.com/Sophia-Gold/21af626e8744b0186ff6c8866c5477c0
Successful boot with X server disabled: https://gist.github.com/Sophia-Gold/966e948870605efc7ee881a16191ebed

Technical details

System: NixOS 17.03
Nix version: 1.11.9
Nixpkgs version: ?
Sandboxing enabled: false

I think you probably already tried it, but just to make sure, did you try disabling both gnome3 and xmonad at the same time, so it's just a plain x session?

Yup. The journalctl dump is in the gist I called "default display manager and xterm." Especially the fact that neither of the likely suspects, gnome and/or gdm, seem to be the cause is really what's puzzling me.

Thought so. :confused:
Anyway, by the way, are you sure that /path/to/.Xmodmap exists? Because if not, it may just be that the displayManager.sessionCommands are not being completed. But then if that was the case, xterm should not be starting.. so I'm unfortunately out of ideas of what it could be, sorry. :disappointed:

Ugh, I feel silly. I'm pretty sure that line is exactly it 🤦🏻♀
Furthermore, I'm pretty sure it should be services.xserver.displayManager.sessionCommand.lib.mkAfter rather than simply services.xserver.lib.mkAfter. I'm not sure where I copied it from.
Can you advise on the proper syntax as well as whether I need to set other sessonCommands? It's not covered in the X Windows section of the manual so this is one of many times I wish the wiki was still extant.

Oh right, well if you don't need to run xmodmap, then you don't need any sessionCommands at all and you can just remove it. :) It's just for running extra commands if you need it, just like an .xinitrc file would, see https://nixos.org/nixos/options.html#displaymanager.sessioncommands.
And that is the correct syntax, the with pkgs; just brings pkgs into scope for the lib.mkAfter ''xmodmap /path/to/.Xmodmap''; part. Although I'm not sure why it's brought into scope, nor what mkAfter even does. :woman_shrugging: With a quick search, it seems like you may have gotten this from https://wiki.haskell.org/Xmonad/Installing_xmonad#NixOS.
There's https://nixos.wiki/ now, but I'm not sure if it's anything like the old wiki.

Hmm... I removed both those lines and still am experiencing the same behavior. After login with either gdm or SLiM I see a blank display manager screen with the cursor frozen :slightly_frowning_face:
I was starting to wonder whether it has to do with a broken gnome extension as in #19269, but journalctl makes it seem like they're all loading fine.
The dumps look about the same towards the end whether or not I disable xmonad and/or gnome; here's one with both enabled: https://gist.github.com/Sophia-Gold/95efcf6a10e83f36e0392e781e05c9b1

It shouldn't be an issue with a gnome extension if it still freezes with just a plain x session, so the following:
{
  services.xserver = {
    enable = true;
    layout = "us";
  };
}

The above should start a plain x session and open an xterm window in it. If this fails, we can probably rule out both xmonad and gnome. But yeah again, I'm assuming you've already tried this and seen that it still freezes as you said before? :confused: Oh yeah and also, you're running nixos-rebuild each time after changing the configuration.nix yeah? (Not to come across as patronizing or anything, just making sure.)

Yup, commenting out of everything but xterm is displaying the same behavior as previously. I should note it does load a prompt, but it doesn't tile to cover the full screen and the cursor and keyboard remain frozen.
My assumption that my regular setup (gdm + gnome3 + xmonad) doesn't get past the display manager is just based on it freezing on a blank screen of the gdm color rather than loading my desktop background.
I'm by no means an expert on debugging with journalctl, but it seems to support this: as you can see in the numerous dumps I posted above, in all the cases when I have X Windows enabled it seems to load fine, connect to WiFi, and then doesn't show any obvious errors while it remains frozen. From what I can tell it never even tries to load a desktop and/or window manager.
And yes, my general practice with changes like this is to run nixos-rebuild boot and then boot from GRUB default (whereas when just adding packages I'll use nixos-rebuild switch). Please do feel free to be patronizing...unless anyone can reproduce this based on my configuration.nix then we're starting to rule out anything other than human error :confused:

I realized I should also mention the odd chance of it being hardware specific (it's an ASUS gaming laptop) as I've had some odd issues with suspend/hibernate/freeze. However, I assumed that was all due to gnome.

Wait, what do you mean commenting out everything but xterm?
And yes from my understanding, it should "load a prompt", i.e. an xterm window, and from memory it's not fullscreen. So if it does get to opening an xterm window, then I believe it means at least it's gone past the display manager.... :thinking:
After it supposedly freezes, can you still access the virtual TTYs by pressing ctrl+alt+F1? Because if you can, then it probably means the keyboard is working okay and it would be something to do with just X then. Also the other thing that could be happening if ctrl+alt+F1 works is that when you try to type in the xterm shell prompt, it simply doesn't display anything because the window may not have focus, but you cannot focus it because you cannot move the mouse, so that might make it seem like the keyboard also doesn't work even though it might.
I also thought this line (from your first dump in the original post of this thread) https://gist.github.com/Sophia-Gold/31d56817b0300cebbc2fbcd6de4e64fa#file-journalctl-gdm-gnome3-xmonad-L1414 could be of interest? And also the last line there as well?
Also the other thing you could try is to get a live usb/disk of a graphical distribution (of any distro) and live boot into it and see if your hardware (mouse and keyboard) work, cause it may even just be that linux doesn't support it yet.. :disappointed: And of course remember to check for the kernel version if your hardware does work (i.e. uname -a) on it, so that you can set that in NixOS. Alternatively you could also just try using a newer kernel and see if anything changes: https://nixos.org/nixos/manual/index.html#sec-kernel-config. For example to get the latest kernel, boot.kernelPackages = pkgs.linuxPackages_latest.
I'm sort of clueless as to what it could be though. :confused:

I meant commenting out everything in services.xserver except the two lines you posted above.
I believe I attempted to launch a TTY from xterm (I certainly have on most of these tests before simply holding down the power button), but should try again and circle back.
I was initially suspicious of those lines from journalctl as well, but (like many things in there) I have little idea what they mean.
As mentioned in the title of this issue, I noticed the change when switching from 16.09 to 17.03. Although, I do understand that's unlikely the cause and more likely a change that occurred simultaneously with it...for example a kernel update. I was under the impression that not specifying a kernel version keeps me up to date with the latest, but that appears not to be the case since my current build is using 4.9.6. I'll update it and circle back if the problem persists.

Ok well I'm sort of out of ideas then for now, will update if I think of anything else, but I think that's probably unlikely. :confused:
But also, "launch a TTY from xterm"? As in you tried running a command in xterm? But I thought you couldn't use xterm?

I meant launching a TYY using the keyboard shortcut, not the tty command that returns the current session.
Can you please confirm what kernel version Nix defaults to when none is specified? Someone on irc specifically told me I'd get the latest as default so I'd like to know whether adding the line you suggested would actually change anything or whether my conflict could be with the latest version (again, the one X Server works fine from is 4.9.6...I suppose I could also try specifying that in a 17.03 build).

I've ruled out a kernel issue. As mentioned, the build that successfully launches X Server has 4.9.6. All problematic builds have had the same subversion (not sure how to specify a patch version in my config) until I recently tried 4.14 to no effect. At least I know the information I was given on irc about Nix defaulting to the latest kernel version was inaccurate.
I also suspected it could be the same issue documented here: https://github.com/NixOS/nixpkgs/issues/23545 as I had xmonad xmonad-contrib and xmonad-extras declared both in environment.systemPackages and windowManager.xmonad.extraPackages. I deleted the former in my config as well as ~./.xmonad yet the issue persisted. I even tried commenting out everything related to xmonad in services.xserver for good measure.
Finally, as far as org.freedesktop.PackageKit I do get some warnings from it on my successful build on lines 1374, 1378, and 1379: https://gist.github.com/Sophia-Gold/00929dd4a77eaed4d8d92a465d4d084d. Afaik, PackageKit is unnecessary when using gnome with Nix and may be the culprit per here: https://github.com/NixOS/nixpkgs/issues/21230 since I'm still on Nix 1.11.9. However, I'm not aware how to disable it (or other freedesktop packages for that matter). Would services.packagekit.enable = pkgs.lib.mkForce false do it?
Alternatively, I wonder if issues with PackageKit were resolved with 1.2? It appears to have been stable for some time now (https://github.com/G-Node/nix/releases), but nix-env upgrade still leaves me with 1.11.6 for 16.09 or 1.11.9 if I call nixos-rebuild switch first to get 17.03.

Yes so right now the linux kernel version is set to 4.9 (the first line in this snippet):
https://github.com/NixOS/nixpkgs/blob/5f0a7cded718d0092229ba078cb7f0f7733bf636/pkgs/top-level/all-packages.nix#L12473-L12478
And using linuxPackages_latest gets you 4.13, which is the latest stable release of linux right now. But of course, what these attributes are actually set to on an installation may be different depending on the NixOS version, since that means a different nixpkgs version.
And yeah, if not even just a plain X session works, it's probably not to do with xmonad (but fair, I know you're just documenting what you've tried).

However, I'm not aware how to disable it (or other freedesktop packages for that matter). Would services.packagekit.enable = pkgs.lib.mkForce false do it?

I don't know how to disable it, but looking at https://gist.github.com/Sophia-Gold/31d56817b0300cebbc2fbcd6de4e64fa#file-journalctl-gdm-gnome3-xmonad-L1414 again (I missed this before), it seems to be a gnome thing (".gnome-shell-wr")? My journalctl doesn't seem to mention org.freedesktop.packagekit anywhere either when I use X with just the herbstluftwm window manager. So it seems like it's probably not the issue then. And even then, looking at its description on wikipedia, it seems pretty useless to Nix (seems to be some sort of front end for package managers, in which none of the backends mention Nix), so it would seem harmless even if it cannot work. But of course you can try that!

Alternatively, I wonder if issues with PackageKit were resolved with 1.2? It appears to have been stable for some time now (https://github.com/G-Node/nix/releases), but nix-env upgrade still leaves me with 1.11.6 for 16.09 or 1.11.9 if I call nixos-rebuild switch first to get 17.03.

I'm assuming you meant 1.12 not 1.2, and https://github.com/NixOS/nix not the neuroscience information exchange format! :stuck_out_tongue_closed_eyes:
But I don't think Nix 1.12 has been released yet. I don't know how to upgrade Nix the package manger itself (it just seems gets upgraded automatically with the rest of the distribution for me), but are you sure it's nix-env upgrade and not nix-env --upgrade? But in any case I think the latest release of Nix is only 1.11.15, and as I said above, packagekit is probably not the issue if it's a gnome thing (sorry for mentioning it initially).

Thank you for clearing up how kernel updates work, even though we've ruled that out as the cause.
And wow, I must have written this before coffee to have linked to Nix, the neuroscience information exchange format :rofl: At least I know my Nix version is up to date...
I agree it really does seem like a gnome issue, specifically with GDBus, and whatever extensions are causing trouble are just still present when I launch into xterm. I'm at loss, though. I know you can specify gnome packages with services.dbus.packages = with pkgs, but most users don't seem to need to.
Fwiw, the issue I linked to above about PackageKit says it's totally unnecessary with Nix and links to another issue with how to fix it until 1.12 hits stable, although it seems to make more sense to just disable it...if services.packagekit.enable = pkgs.lib.mkForce false does in fact do that.

I don't use version control on my configuration.nix that often, but looking at my commits now it appears I did switch to gdm right around the time this all broke. Now that I'm starting to realize commenting things out and then rebuilding often doesn't delete everything they've installed I wonder if that could be it.

launch into xterm

I keep getting a bit confused, you mean launched into X right? Because xterm is the terminal emulator. But anyways, so if packagekit is related to gnome, but if the issue still happens with just:
{
  services.xserver = {
    enable = true;
    layout = "us";
  };
}

which should not be using gnome at all, then packagekit should not be the cause to your freezing problem.

Fwiw, the issue I linked to above

Oh right I missed this :see_no_evil:, so packagekit does have a Nix backend. So I guess to disable it you could also try:
{
  nixpkgs.config.packageOverrides = pkgs: {
    packagekit = pkgs.packagekitoverride {
      enableNixBackend = false;
    };
  };
}

Now that I'm starting to realize commenting things out and then rebuilding often doesn't delete everything they've installed I wonder if that could be it.

I'm assuming you're referring to unused /nix/store paths? (i.e. the garbage that can be collected with nix-collect-garbage.) That shouldn't be an issue at all. In fact, this not being an issue is like the whole point of declarative configuration. :stuck_out_tongue:

I was referring to things like how in the issue I linked to about xmonad it appeared to be the config file, ~/.xmonad, hanging around that was causing the problem so it seems like sometimes one does need to manually clean things. At least this is my current hypothesis to explain why the fact that xterm freezes after loading could be due having once built gdm or PackageKit or whatever else. Otherwise, I truly have no idea what's going on here.

Yes so ~/.xmonad being a program configuration file that xmonad uses, it will obviously affect things if xmonad is enabled. But if not enabled, then the ~/.xmonad file laying around should not be affecting things. So similarly, if you only have the enable = true; and layout = "us"; attributes for services.xserver, it means that gnome is not enabled, and therefore any such program files laying around from a previous installation of gdm or packagekit should not affect anything. (Because iirc the default display manager is slim.)

No idea what's going on when I boot to xterm then. I'll try disabling PackageKit just for the sake of it when I'm able to reboot, but am not particularly hopeful it will have any effect.

Hmm, except for the gdm case (which gets struck earlier), I think the real error line is

.../bin/xauth: file /home/sophia/.Xauthority does not exist

You could try some proposals found on the internet (like these), but I would first try a different DM – I have personally good experience with lightdm.  GDM tends to be difficult to package well in NixOS and wasn't much used/maintained by NixOS people around 17.03 AFAIK; slim is the default but that's been heavily disputed as it's been unmaintained upstream for many years now.

@vcunat thanks for chiming in!
Lightdm got my hopes up because it didn't freeze like GDM and SLiM, but still didn't launch Gnome or XMonad. I was also able to use ctrl-alt-1, surely able to just being able to use the keyboard/touchpad in general, although it just gave me a black screen with no TTY prompt. Ctrl-alt-7 didn't bring me out of it as well.
Here's the journalctl dump from that: https://gist.github.com/Sophia-Gold/71858c4a6f15ab37d1c285ffa047c3ba. It doesn't have that xauth error, although does still complain about PackageKit (I haven't tried disabling it yet).
I should also probably mention the 16.09 build I'm able to boot into does use GDM (I'm not sure how to check the version to see if it changed with the NixOS upgrade since I don't have it installed in systemPackages).

Lightdm got my hopes up because it didn't freeze like GDM and SLiM, but still didn't launch Gnome or XMonad. I was also able to use ctrl-alt-1, surely able to just being able to use the keyboard/touchpad in general, although it just gave me a black screen with no TTY prompt.

XMonad doesn't open a terminal or do anything by default, you have to use keybindings to open a terminal, but I'm assuming you knew that? (Just making sure you know that xmonad is indeed not being launched.) Not sure about what happens when using xmonad with gnome though.
Anyway so this might mean that a plain X session is not the problem, and the problem might be to do with either xmonad or gnome (if it isn't the lack of the .Xauthority file).

Yup. I tested all iterations of my X Server services when opening this issue.
The problem really does appear to be PackageKit at this point. I tried disabling it with the override you suggested above, but it doesn't seem to have worked since I still get the same errors (this is back with GDM btw): https://gist.github.com/Sophia-Gold/9fb518967b9ef0264575e987ea173724.
Also, apologies but it appears I was looking at the wrong date in Git when I suspected GDM. The last build I can boot into, and the last using 16.09, was from May 22nd. Here's the log for my configuration.nix: https://github.com/Sophia-Gold/configuration.nix/commits/master. The only changes committed on that day and the following one involve disabling XMonad and adding some trivial packages.

Eww, kernel errors related to the nouveau driver.  You can try to either (1) use the unfree driver as it tends to work better overall services.xserver.videoDriver = "nvidia"; or (2) use a different kernel version, hopefully free from that bug.  16.09 used 4.4.x kernel and those will still be supported upstream for about half a year, or you may try a newer one, but there's no longterm-support version newer than the default 4.9.x yet so you'd have to be switching over time (to 4.13 ATM).

The unfree driver breaks X entirely for me. It boots into a tty and the keyboard is glitchy and barely responsive :disappointed:

Oh, wait. Also I was on 4.9 even when using 16.09 and tried 4.13 on 17.03. Neither changed anything. Should I try 4.13 with the unfree driver?

Hmm, strange that it's breaking so much.  Yes, I would probably try 4.13 with both unfree and the free driver (separately).

Same behavior (no display manager, straight to tty) with boot.kernelPackages = pkgs.linuxPackages_latest. I tried that previously without specifying the unfree driver and saw no changes.

I got a bit fatigued in troubleshooting this, but had to upgrade to 17.09 for one package so was hoping that would help (it didn't). At this point, I'm pretty sure it's a hardware specific issue with the graphics card...especially since I'm unable to disable either the Nvidia card or integrated graphics in BIOS. I need a new laptop anyway :slightly_frowning_face:

I also appear to be missing a ton of packages after upgrading to 17.09, including google-chrome and my password manager. Lovely...

Perhaps then look into ways of running hardware tests for your laptop, if possible?
I don't know why you would be missing packages though. :( To begin with, maybe try setting your environment.systemPackages to only include google-chrome (temporarily) and see if that still works or not.

Well, I was able to install pass with nix-env but obviously I can't do that with Chrome unless I set up a separate config.nix for allowUnfree. I'm still using Chrome now, but it's 55.
@7c6f434c was very helpful on irc and suspects this machine may just be screwed as far as graphics goes, which I seemed to have confirmed from the BIOS.

For Chrome you can run NIXPKGS_ALLOW_UNFREE=1 nix-env -iA google-chrome…

I mostly said that everything with Nvidia is partially messed up as far as graphics goes…
I would also try with videoDrivers = lib.mkForce ["modesetting"]; maybe.

I thought you also mentioned something to the effect of this laptop possibly having graphics wired in an odd manner, which would explain my shock at learning I have no control over it in the BIOS.
The actual specs refer to the GeForce GTX 960M as a "graphics coprocessor," which makes me wonder if they're just using that term for marketing or if the motherboard is actually designed so it replaces integrated graphics. It seems like possibly the latter, which is kind of horrible.
I'm constantly reminded that reinstalling from scratch is pointless with NixOS, but since I have no idea what upgrade in the 17.03 channel led to this I'm tempted to just boot from a disk image and add in bits of my configuration.nix until I find it. I suppose bisecting it would achieve the same ends, but considering I've tried all combinations of the likely culprits and never even been able to boot into just an xterm I'm suspicious some of the garbage left behind when removing packages could be misleading when troubleshooting.

It is possible.  My old notebook has both integrated GPU (i3) and dedicated one (nvidia), and it's only able to use the dedicated one, on any OS.

I would expect that if an Asus system cannot switch off Nvidia GPU, it
has no other GPU wired to the video outputs.
Booting from a CD might help in terms of making sure… but maybe some
special tricks are necessary with that video card, then you need to find
a generic description and then adapt it to NixOS…

Well, it worked before I switched channels to 17.03 so I can probably fix it as long as I can isolate exactly which package(s) I need to override to an earlier version. I guess I'll see if I get around to that before I get around to purchasing a new laptop...

(triage) Is this still an issue with more recent releases?

(triage) is this still an issue?


Network manager topology add tags

@epwinchell hello, I'll need also tag.png hexa code for the topology. Can you send me that, please?

@ladas I don't understand what you need. You should be able to use the "fa-tag"

@epwinchell right, but I haven't found it here https://www.patternfly.org/styles/icons/
the topology chart is using the hexadecimal codes directly e.g. like this https://github.com/Ladas/manageiq/blob/35eb80522bec8d410f199f169ffc41807579cb2a/app/services/container_topology_service.rb#L24-24

@Ladas Try this:  fa-tag []  found here: https://fortawesome.github.io/Font-Awesome/cheatsheet/

@epwinchell thank you :-)

@miq-bot add_tag topology

@miq-bot add_label topology

@miq-bot remove_label wip


[Java] Avoid memory allocation at varDataDisplay

varDataDisplay - use method getVarFieldName(Appendable) - no memory allocation. unlike calling method varFieldName where it allocates memory (new byte[] and new String).

This generates code which does not compile.

This generates code which does not compile.

yep, fixing it. apperantly code differs for UTF and ASCII.

The UTF-8 implementation allocates too. I'm not in favour of supporting this as it is too brittle.

The UTF-8 implementation allocates too. I'm not in favour of supporting this as it is too brittle.

I'm using at my project ASCII chars and I'm willing to use the method appendTo that sits on the generated xxxDecoder class(es), which is very useful for logging incoming messages.

I've pushed a more obvious changes which saves on allocation.

I've pushed a more obvious changes which saves on allocation.

seen your commit and it paratially solves the problem see my comment there
https://github.com/real-logic/simple-binary-encoding/commit/391527ac04df032671167b12d923b29dad30c5c0#r35648950

A better approach for you to take is to use the append version if and only iff the encoding is ASCII otherwise output as is at present.


Memory leak due to recursion when using 'subscribe'

The subscribe method on the client extension is referencing itself instead of the instance method on FalClient which causes a recursion loop creating infinite number of mallocs. Removing the new method from the extension and only leaving the deprecated pointing to the new method solves the issue.

Hey @okaris thanks for reporting this and doing the deep dive.
The 0.5.1 tag was incorrect and had this line https://github.com/fal-ai/fal-swift/blob/0.5.1/Sources/FalClient/FalClient.swift#L49 that would cause the issue. I still don't understand how the tag had it, even though it pointed to the correct commit: https://github.com/fal-ai/fal-swift/commit/133bdfa4ee8d4ab34e777ad352ec2b1a5ab934e7#diff-59b0e362f18b1b0ca6c551334fd567b841e3b51381af054135267d416014374dL49
Anyway, I apologize for the frustration. The tag 0.5.2 was created and contains the correct implementation.

Closing this, as this was fixed as of version 0.5.2


validator: check signatures against PKI

rules.json format was updated:

a pki property must be specified: it defines a list of "identities" linked to public keys. It allows signature validation rules to be more user-friendly (one can require signatures by name, role or public key)
an identifier id must be specified in a validator
the signatures property of a validator is now a slice of strings. In the example below, the init action of the testProcess process (the first segment) needs to be signed by both TESTKEY1 (alice) and TESTKEY2 (bob). If this property is omitted or empty, no signature is required. Nevertheless, it there are signatures in link.Signatures, they will still be verified and an error will be thrown if an error is encountered.

{
  "pki": {
    "TESTKEY1": {
      "name": "Alice Van den Budenmayer",
      "roles": ["employee"]
    },
    "TESTKEY2": {
      "name": "Bob Wagner",
      "roles": ["manager", "it"]
    }
  },
  "validators": {
    "testProcess": [
      {
        "id": "testProcess-schema",
        "type": "init",
        "signatures": ["manager", "alice van den bundemayer"],
        "schema": {
          "type": "object",
          "properties": {
            "string": {
              "type": "string"
            }
          }
        }
      }
    ]
  }
}

This change is

Great work!

Reviewed 18 of 18 files at r1.
Review status: all files reviewed at latest revision, 4 unresolved discussions.

validator/configloader.go, line 80 at r1 (raw file):

}

// loadPKIConfig deserializes json into a PKI struct. It checks that public keys

truncated comment apparently :)

validator/configloader.go, line 83 at r1 (raw file):

func loadPKIConfig(data json.RawMessage) (*PKI, error) {
	if len(data) == 0 {
		return nil, ErrNoPKI

Does this mean that even if I don't require signatures I must specify a pki?

validator/pki.go, line 57 at r1 (raw file):

}

func (pv pkiValidator) matchRequirement(requirement, publicKey string) bool {

nit: this could probably be delegated to PKI so that we don't leak the logic related to Names, Roles...

validator/pki.go, line 87 at r1 (raw file):

// Validate checks that the provided dignatures match the required ones.
// a requirement can either be : a public key, a name defined in PKI, a role defined in PKI

nit: no space before a ":" in english and comments should end with ".".

Comments from Reviewable

Review status: all files reviewed at latest revision, 4 unresolved discussions.

validator/configloader.go, line 80 at r1 (raw file):
Previously, such (Adrien Montfort) wrote…
truncated comment apparently :)

Done.

validator/configloader.go, line 83 at r1 (raw file):
Previously, such (Adrien Montfort) wrote…
Does this mean that even if I don't require signatures I must specify a pki?

yes indeed. I see why it can be misleading, I'm changing this so that the pki field is required only when specifying a signatures property in a validator.
Done.

validator/pki.go, line 57 at r1 (raw file):
Previously, such (Adrien Montfort) wrote…
nit: this could probably be delegated to PKI so that we don't leak the logic related to Names, Roles...

Done.

validator/pki.go, line 87 at r1 (raw file):
Previously, such (Adrien Montfort) wrote…
nit: no space before a ":" in english and comments should end with ".".

Done.

Comments from Reviewable

Reviewed 17 of 18 files at r1, 3 of 3 files at r2.
Review status: all files reviewed at latest revision, 8 unresolved discussions.

cs/cstesting/cstesting.go, line 146 at r1 (raw file):

// The key must be an instance of ed25519.PrivateKey
func SignLinkWithKey(l *cs.Link, priv ed25519.PrivateKey) *cs.Link {
	pub := priv[32:]

priv.Public() looks a bit less obscure to obtain the public key :)

validator/configloader.go, line 31 at r1 (raw file):

	// ErrBadPublicKey is returned when a public key is empty or not base64-encoded
	ErrBadPublicKey = errors.New("Public key must be a non-null base64 encoded string")

nit: usually go error messages are lower case without punctuation (supposedly easier to grep)

validator/configloader.go, line 43 at r1 (raw file):

// LoadConfig loads the validators configuration from a json file.
// The configuration returned can be then be used in NewMultiValidator().

nit: typo multiple "be"

validator/multivalidator.go, line 55 at r1 (raw file):

		v = append(v, newPkiValidator(pkiCfg))
	}
	defaultValidator := newSignatureValidator(&signatureValidatorConfig{})

I'm not a big fan of this choice. The way I would have done it is that schemaValidators and signatureValidators are both validator instances that are added to the list of validators inside the multi-validator, and there is not notion of important or default validator.
It feels more flexible for adding future types of validators we could need in the future.
But it is your implementation so it's your choice :), don't hesitate to come discuss it IRL, there might be implementation details that make my suggestion complicated in practice.

Comments from Reviewable

Review status: all files reviewed at latest revision, 8 unresolved discussions.

cs/cstesting/cstesting.go, line 146 at r1 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
priv.Public() looks a bit less obscure to obtain the public key :)

Done.

validator/configloader.go, line 31 at r1 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
nit: usually go error messages are lower case without punctuation (supposedly easier to grep)

Done.

validator/configloader.go, line 43 at r1 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
nit: typo multiple "be"

Done.

validator/multivalidator.go, line 55 at r1 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
I'm not a big fan of this choice. The way I would have done it is that schemaValidators and signatureValidators are both validator instances that are added to the list of validators inside the multi-validator, and there is not notion of important or default validator.
It feels more flexible for adding future types of validators we could need in the future.
But it is your implementation so it's your choice :), don't hesitate to come discuss it IRL, there might be implementation details that make my suggestion complicated in practice.

As discussed IRL, I'll keep this for now and I'll organise a meeting to discuss the following points:

do we want to add boolean algebra in validators (eg: this link in valid in case X OR Y) ?
do we want to have special behaviours for validators (eg: default validator, fallback) ?
what's in the scope of the validation language, or in the rules.json structure ?
do we want validators to work across different processes ?
what's the right balance between complexity and flexibility, and to which extent does the process designer prevents a user from introducing security flaws ?

Comments from Reviewable

Reviewed 2 of 2 files at r3.
Review status: all files reviewed at latest revision, 5 unresolved discussions.

validator/multivalidator.go, line 55 at r1 (raw file):
Previously, simonvadee wrote…
As discussed IRL, I'll keep this for now and I'll organise a meeting to discuss the following points:

do we want to add boolean algebra in validators (eg: this link in valid in case X OR Y) ?
do we want to have special behaviours for validators (eg: default validator, fallback) ?
what's in the scope of the validation language, or in the rules.json structure ?
do we want validators to work across different processes ?
what's the right balance between complexity and flexibility, and to which extent does the process designer prevents a user from introducing security flaws ?

Sounds good, thanks !

Comments from Reviewable

Reviewed 2 of 2 files at r4.
Review status: all files reviewed at latest revision, 4 unresolved discussions.

Comments from Reviewable

Reviewed 4 of 4 files at r5.
Review status: all files reviewed at latest revision, 4 unresolved discussions.

Comments from Reviewable

@t-bast @such @alexppxela I just realised one thing : shouldn't a PKI be bound to a single process ? A single public key might not have the same permissions (role) across processes right ? I propose :
{
  "pki": {
    "testProcess": {
      "TESTKEY1": {
        "name": "Alice Van den Budenmayer",
        "roles": ["employee"]
      },
      "TESTKEY2": {
        "name": "Bob Wagner",
        "roles": ["manager", "it"]
      }
    }
  },
validators: { ... }
}```

what do you think ?

Review status: all files reviewed at latest revision, 4 unresolved discussions.

Comments from Reviewable

Reviewed 9 of 18 files at r1, 1 of 3 files at r2, 1 of 2 files at r3, 1 of 2 files at r4, 3 of 4 files at r5.
Review status: all files reviewed at latest revision, 8 unresolved discussions.

tmstore/tmstore_test.go, line 98 at r5 (raw file):

	})

	t.Run("Signature validation failed", func(t *testing.T) {

A validation success test is missing, if it's possible

validator/configloader.go, line 31 at r1 (raw file):
Previously, simonvadee wrote…
Done.

It is for log encapsulation which already contains sentence formatting

validator/multivalidator.go, line 55 at r5 (raw file):

		v = append(v, newPkiValidator(pkiCfg))
	}
	defaultValidator := newSignatureValidator(&signatureValidatorConfig{})

For me there is no interest to have a defaultValidator (or I missed something), it is just possible to append this one in validators
I think we will discuss about this tomorrow, this is one of your meeting point

validator/schema.go, line 55 at r5 (raw file):

// schemaValidator validates the json schema of a link's state.
type schemaValidator struct {
	*schemaValidatorConfig

Hum, a validator is not a config, it has a config.

Comments from Reviewable

It's a good point. We might want a user to have different roles in different processes.
But we might also want to be able to say : this user has this role for all processes.
Let's discuss this further tomorrow at the meeting, we need to formalize this correctly.

Review status: all files reviewed at latest revision, 7 unresolved discussions.

validator/multivalidator.go, line 55 at r5 (raw file):
Previously, alexppxela (Alexandre Thibault) wrote…
For me there is no interest to have a defaultValidator (or I missed something), it is just possible to append this one in validators
I think we will discuss about this tomorrow, this is one of your meeting point

Exactly :) but Simon has a good explanation for this defaultValidator, so it would be good to discuss it together.

validator/schema.go, line 55 at r5 (raw file):
Previously, alexppxela (Alexandre Thibault) wrote…
Hum, a validator is not a config, it has a config.

I agree, it's confusing.

Comments from Reviewable

:+1:

Review status: all files reviewed at latest revision, 7 unresolved discussions.

tmstore/tmstore_test.go, line 98 at r5 (raw file):
Previously, alexppxela (Alexandre Thibault) wrote…
A validation success test is missing, if it's possible

it is a pain but it should be possible, I'll try and add this.

validator/multivalidator.go, line 55 at r5 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
Exactly :) but Simon has a good explanation for this defaultValidator, so it would be good to discuss it together.

Actually , I HAD a good explanation for it, but now that PkiValidator and SignatureValidator are two distinct validators and that SignatureValidator.ShouldValidate() always return true, it does not make any difference to append it at the end of validatorsor to have a defaultValidator. Since we all are on the same page about this, I'm updating it.

validator/schema.go, line 55 at r5 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
I agree, it's confusing.

It allowed me to directly access all the underlying properties of a validator method, but I understand it can lead to confusions, I'll change this. I'll actually remove entirely the config part, since it does not seem useful at all.

Comments from Reviewable

Good idea, or I prefer configuration inheritence but without a good UI it should be confusing

Reviewed 1 of 18 files at r1, 1 of 2 files at r4, 1 of 4 files at r5.
Review status: all files reviewed at latest revision, 4 unresolved discussions.

Comments from Reviewable

Review status: 3 of 19 files reviewed at latest revision, 6 unresolved discussions.

tmstore/tmstore_test.go, line 98 at r5 (raw file):
Previously, alexppxela (Alexandre Thibault) wrote…
That's why I added "if it is possible". If a test is too complicated or unreadable, it's is a pain to be maintained.

Done.

validator/multivalidator.go, line 55 at r5 (raw file):
Previously, simonvadee (Simon Vadée) wrote…
Actually , I HAD a good explanation for it, but now that PkiValidator and SignatureValidator are two distinct validators and that SignatureValidator.ShouldValidate() always return true, it does not make any difference to append it at the end of validatorsor to have a defaultValidator. Since we all are on the same page about this, I'm updating it.

Done.

validator/schema.go, line 55 at r5 (raw file):
Previously, simonvadee (Simon Vadée) wrote…
It allowed me to directly access all the underlying properties of a validator method, but I understand it can lead to confusions, I'll change this. I'll actually remove entirely the config part, since it does not seem useful at all.

Done.

Comments from Reviewable

Reviewed 1 of 16 files at r6.
Review status: 4 of 19 files reviewed at latest revision, 5 unresolved discussions.

validator/multivalidator.go, line 35 at r6 (raw file):

// The configuration should be loaded from a JSON file via validator.LoadConfig().
func NewMultiValidator(path string) (Validator, error) {
	validators, err := loadConfig(path)

We explicitly wanted to decouple config loading from the validator itself :)
It's more flexible (especially for tests) if this takes directly a config object as input.
It also allows different ways of loading/creating the config.

Comments from Reviewable

Review status: 2 of 19 files reviewed at latest revision, 5 unresolved discussions.

validator/multivalidator.go, line 35 at r6 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
We explicitly wanted to decouple config loading from the validator itself :)
It's more flexible (especially for tests) if this takes directly a config object as input.
It also allows different ways of loading/creating the config.

Done.

Comments from Reviewable

It's in good shape now, thanks for all the changes ! A couple comments and it should be good for me.

Reviewed 7 of 16 files at r6, 10 of 10 files at r7.
Review status: all files reviewed at latest revision, 8 unresolved discussions.

validator/configloader_test.go, line 132 at r7 (raw file):

		assert.NotNil(t, validators)

		assert.Len(t, validators, 5)

In all these tests, now that there are multiple validator types, it would be good to assert on the type of validators created. In this test it would be nice to verify that there are both schema validators and pki validators, and in tests below where there's no signature it would be nice to verify that the validator created is a schema validator.

validator/multivalidator.go, line 33 at r7 (raw file):

// NewMultiValidator creates a validator that will simply be a collection
// of single-purpose validators.
// The configuration should be loaded from a JSON file via validator.LoadConfig().

nit: this comment line is outdated

validator/pki.go, line 98 at r7 (raw file):

		}
		if !fulfilled {
			return errors.Errorf("Missing signatory for validator %s: signature from %s is required", pv.Config.ID, required)

nit: signatory

validator/validator.go, line 36 at r7 (raw file):

// ChildValidator defines an interface that have to be implemented by a custom validator (eg: signature, schema, state...)
type ChildValidator interface {

I don't see the need for a new interface (especially in the latest revision).
I think it makes sense to add your ShouldValidate method directly to the validator interface.
This way we can imagine that a MultiValidator can have another MultiValidator as a child (ValidatorCeption !).

Comments from Reviewable

Thank you for the thorough feedback !

Review status: all files reviewed at latest revision, 8 unresolved discussions.

validator/configloader_test.go, line 132 at r7 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
In all these tests, now that there are multiple validator types, it would be good to assert on the type of validators created. In this test it would be nice to verify that there are both schema validators and pki validators, and in tests below where there's no signature it would be nice to verify that the validator created is a schema validator.

You're right ! i'll update that tomorrow.

validator/multivalidator.go, line 33 at r7 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
nit: this comment line is outdated

Done.

validator/pki.go, line 98 at r7 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
nit: signatory

I think it's accurate (it means "signataire"), do you think signature is clearer ?

validator/validator.go, line 36 at r7 (raw file):
Previously, t-bast (Bastien Teinturier) wrote…
I don't see the need for a new interface (especially in the latest revision).
I think it makes sense to add your ShouldValidate method directly to the validator interface.
This way we can imagine that a MultiValidator can have another MultiValidator as a child (ValidatorCeption !).

Done.

Comments from Reviewable

Review status: 11 of 19 files reviewed at latest revision, 8 unresolved discussions.

validator/configloader_test.go, line 132 at r7 (raw file):
Previously, simonvadee (Simon Vadée) wrote…
You're right ! i'll update that tomorrow.

Done.

Comments from Reviewable

Reviewed 7 of 16 files at r6, 2 of 10 files at r7, 8 of 8 files at r8.
Review status: all files reviewed at latest revision, 8 unresolved discussions, some commit checks failed.

Comments from Reviewable

Thanks for all the iterations!

Reviewed 6 of 8 files at r8.
Review status: all files reviewed at latest revision, 5 unresolved discussions, some commit checks failed.

validator/pki.go, line 98 at r7 (raw file):
Previously, simonvadee (Simon Vadée) wrote…
I think it's accurate (it means "signataire"), do you think signature is clearer ?

Didn't know that :)
It's fine then if it's not a typo!

Comments from Reviewable


Handling large data sets with pagination or scrolling.

Is there a strategy for handling large data sets?

I too am interested, doing things like http://stackoverflow.com/questions/8970362/setting-a-max-height-on-a-table feels ugly.

Naw, I wouldn’t use overflow for this. It’s a real anti-pattern on some mobile devices.
I don’t see why you couldn’t use this with pagination links. We don’t have any built in styles for them, but I’d certainly accept a PR that had that included.
Otherwise, this might not be helpful to you but we did develop an extension for https://github.com/filamentgroup/fixed-sticky that worked with table headers. I put in a little bit of time to get this working on busyness.io but we haven’t open sourced the code yet. The code for this is here: https://github.com/zachleat/BUSYNESS/blob/master/public/javascripts/lib/tables.stickyheaders.js if you want to try to get it working but I won’t provide support for this until it’s officially open sourced. :)

In my opinion tablesaw has some really useful features like toggle and swipe which DataTables.net doesn't, but the pagination is a really huge advantage of DataTables.

I triple this, sticky table headers with sticky columns would be awesome

This repository is now using lodash style issue management for enhancements. This means enhancement issues will now be closed instead of leaving them open.
The enhancement backlog can be found here: https://github.com/filamentgroup/tablesaw/issues?utf8=✓&q=label%3Aneeds-votes+sort%3Areactions-%2B1-desc+
Don’t forget to upvote the top comment with 👍!


[FeatureRequest] add option to ignore invalid configuration files

Is your feature request related to a problem? Please describe.
We write from our app to a configfile. (through a tempfile and rename that after serialization was successfull). but even with this precautions we sometimes (once in a year) end up with an empty file while the app is starting.
this empty file prevents the app from starting as the jsonconfigfileprovider has no option to swallow errors.
Describe the solution you'd like
the file is marked as optional. My opinion is that if an optional file is invalid it should be ignored. If it is not possible to implement this behavior with the optional flag there should be another flag that allows to configure this behavior.
Describe alternatives you've considered
currently I use this methods to determine if I should call addjson:
        public static bool ContainsValidJson(this string path)
        {
            if (!File.Exists(path))
            {
                return false;
            }

            using (var fs = new FileStream(path, FileMode.Open))
            using (var sreader = new StreamReader(fs))
            {
                return sreader.IsValidJson();
            }
        }

        public static bool IsValidJson(this TextReader text)
        {
            try
            {
                using (var jreader = new JsonTextReader(text))
                {
                    JObject.Load(jreader);

                    return true;
                }
            }
            catch (Exception)
            {
                return false;
            }
        }

in program.cs
            var app = new WebHostBuilder()
                .ConfigureAppConfiguration( ( host, config ) =>
                {
                    if (path.ContainsValidJson())
                    {
                        config.AddJsonFile(path, optional: true);
                    }
                } )

@HaoK What do you think of this? I could see us having optional also mean empty, ignore if empty or not exists. But I don't think I'd stretch the single bool to mean invalid config in general.
@JanEggers It seems unusual that a situation where an error somewhere else failed to generate a config file, but the system should still start and run fine. Can you share what sort of stuff is in the config file or more details of the scenario?
In general I am not sure that this is a common enough situation that we should potentially surprise someone with the suggestion implementation. Happy to be persuaded by more examples of scenarios where it's compelling though. I am struggling to see them.

There is a feature already to swallow exceptions, you just need hook the load exception, i.e. https://github.com/aspnet/Extensions/blob/master/src/Configuration/test/Config.FunctionalTests/ConfigurationTests.cs#L105
            configurationBuilder.AddJsonFile(c =>
            {
                c.Path = Path.Combine(_fileSystem.RootPath, _jsonFile);
                c.OnLoadException = e =>
                {
                    e.Ignore = true;
                    var exception = e.Exception as FileNotFoundException;
                    Assert.NotNull(exception);
                };
            });

@glennc: Can you share what sort of stuff is in the config file or more details of the scenario?
the file is just completely empty in case something went wrong
@HaoK : thx for pointing that out.
you should look at your tests though because without
c.ResolveFileProvider();
it didnt do anything for me so I dont know if the test is doing anything....
I think this method should include onLoadException as parameter (the other methods can fallback to pass null) to increase visibility of this feature.

also it would be nice if the callback OnLoadException would provide access to DI so I could get a logger to log the error


Using static method while querying relationships breaks the relationship constraint

Hi,  about  https://adonisjs.com/docs/4.1/relationships#_querying_data, it seems that using the static method (https://adonisjs.com/docs/4.1/lucid#_static_methods) instead of fetch() breaks the constraint, for example:
const posts = await user.posts().fetch()
would return user's posts while:
const posts = await user.posts().ids()
would return an array of all posts ids, not only the ones belonging to user.
Package version
"adonis-version": "4.1.0",
"@adonisjs/lucid": "^5.0.3",
Node.js and npm version
v8.0.0
6.1.0

This issue was moved to adonisjs/adonis-lucid#358


Fix new partitioned table

[This is a re-do of the pull request #3366 to fix the author email address to use my google.com email]
I moved the partitioning_type check into the if / elif block to avoid a ValueError that should not have been thrown.
See:
https://cloud.google.com/bigquery/docs/creating-partitioned-tables#creating_a_partitioned_table and the "API" example.
Before my change, the following code failed with the ValueError thrown by this function, but with my change, I was able to successfully create the partitioned table:
destination_table = dataset.table(table_shortname)

if not destination_table.exists():
    destination_table.partitioning_type = 'DAY'
    destination_table.create()

# Use a table partition
destination_partition = dataset.table(table_shortname + '$' + partition_date)

@dhermes @jonparrott
I resubmitted my changes in a new pull request to make sure it has my @google.com email, and withdrew pull request #3366

Thanks @richkadel. This LGTM.
@tswast Any issues?

@richkadel Your code snippet above never uses destination_partition. Is there a typo somewhere?

@dhermes re: @richkadel Your code snippet above never uses destination_partition. Is there a typo somewhere?
No typo. The code that uses the partition itself is not shown. I left the line in to show that the table name used in the "create" call is the table name without the partition suffix. Once the table is created then you can use destination_partition value as the "job_destination" setting to materialize data into just that partition.

@richkadel Gotcha, I just made an edit to make that clearer.

Updates have been pushed and all tests pass.


Build fails because of removeLoggers

Version
nuxt-security:  1.0.0-rc.4
nuxt:  3.8.2
Reproduction Link

https://stackblitz.com/edit/nuxt-starter-tsqs5c?file=README.md
Steps to reproduce
Open Stackblitz repro
Observe the test function in app.vue : it has a console.log that works fine in dev mode (yarn dev)
What is Expected?
App should build successfully
What is actually happening?
Build fails with Unexpected token (Note that you need plugins to import files that are not JavaScript)

Another variant can also happen in ts files : [vite:build-import-analysis] Parse error @:1:1
I'm not providing a repro because it would be too long to set up, but screenshot below on a Pinia store file:

Bug Analysis
The bug happens when console.log is removed from a lambda void-returning arrow-syntax function with a single instruction.
I guess the parser is a bit too aggressive and transforms :
const test = () => console.log('Test')

Into
const test = () =>

Which is invalid Javascript
Solution
This is not a Nuxt-Security bug, this comes from unplugin-remove
Upstream Issue opened at : https://github.com/Talljack/unplugin-remove/issues/3
Temporary Solution for Nuxt-Security : If you encounter this issue, you can either :

set removeLoggers: false
refactor above code to const test = () => { console.log('Test') }

PR #311 fixes


Viewport camera clears the whole window

Bevy version
0.11
[Optional] Relevant system information
AdapterInfo {
  name: "NVIDIA GeForce RTX 2060",
  vendor: 4318,
  device: 7944,
  device_type: DiscreteGpu,
  driver: "NVIDIA",
  driver_info: "536.40",
  backend: Vulkan
}

What you did
Creating two cameras; a viewport camera being drawn after a main one that rendered to the full window. I set the clear color of the camera with the viewport setting expecting it to clear only its viewport area of the window.
What went wrong
The whole window was cleared, even though the scene is limited to the viewport.
Additional information
The documentation does not mention that you must set the viewport cameras to ClearColorConfig::None, so I presume this isn't the expected behavior.
https://docs.rs/bevy/latest/bevy/render/camera/struct.Viewport.html
Minimal reproduction:
bevy_viewport_clear.zip

Can confirm this is happening to me as well.


Simplified @stringfilter decorator and Library with unwrap().

While looking at ticket-6135 (I'm not sure if that ticket is still relevant today), I noticed that Django was manually keeping track of wrapped functions by setting a _decorated_function attribute.
I suspect this was done at a time when functools.wraps didn't exist, or maybe it behaved differently.
Nowadays we can use inspect.unwrap() to retrieve the innermost function object when needed.
And most of the uses of _decorated_function were to access the original __name__ which is not needed because @functools.wraps automatically sets that attribute correctly.

@bmispelon Thanks :+1:


config: s/annotations/labels/

Let's consistently refer to labels as labels so as not to confuse them
with the other kinds of annotations.
Signed-off-by: Jonathan Boulle<EMAIL_ADDRESS>

dup of #464  :)

welp, thanks.


refactor(iroh-willow): refactor peer manager so that proptest passes reliably

Description
Fixes #2695

Refactor peer manager to really keep track of all connections, the previous logic of a single peer state was flawed for simultaneous accepts while closing previous connections.
Better debuggability of reconciler
Add proptest from #2695 and refactor to run in both directions simultaneously.

Breaking Changes

Notes & open questions

Change checklist

[ ] Self-review.
[ ] Documentation updates following the style guide, if relevant.
[ ] Tests if relevant.
[ ] All breaking changes documented.

Will take this over now :v:


Update templates and pipelines for linux deploys

Fixes #minor
Proposed Changes
Fix Python templates and pipelines for linux deploys.
Fix JS templates and pipelines for linux deploys.

Replaced by #3817


Create and log a useful user and/or debug message in src/ramstk/Utilities.py.

This issue was automatically generated from a TODO comment in the file src/ramstk/Utilities.py.

It's unknown what this issue is even referring to.  This can not be implemented.


Thanks for the information

from last year onwards , I used  windows powershell  cmd , to  export the custom startmenu layout.
Before exporting , I did pinned some applications  and  unpinned some  apps for my  custom design.
I will ask  another question after two days .
Thanks for the detailed explanation . Thanks to the  authors

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 79fccecd-5484-0304-d82b-9425f1889a22
Version Independent ID: 4df6b138-c1b5-b551-22c9-19d3d1f9105d
Content: Customize and export Start layout (Windows 10)
Content Source: windows/configuration/customize-and-export-start-layout.md
Product: w10
Technology: windows
GitHub Login: @jdeckerMS
Microsoft Alias: jdecker

@officedocsbot assign @mypil

@RAJU2529 - Thank you for submitting feedback.
Is there something within this article that you suggest to be added or edited? Please share more details.
Thank you.

Yes madam. Tomorrow I post with certain doubts.

@RAJU2529 - I would like to follow up on this. Is there something within this article that you suggest to be added or edited? Please share more details.
Thank you.

Default location of Start menu layout
C:\Users\Default\AppData\Local\Microsoft\Windows\Shell
C:\Users\username\AppData\Local\Microsoft\Windows\Shell\
I would  like to like  add the  full command  like this for exporting
Export-StartLayout -Path C:\StartMenuLayout.xml
I would  like to like  add the  full command  like this for importing m open the cmd prompt run as admin
copy /y C:\StartMenuLayout.xml C:\Users\Default\AppData\Local\Microsoft\Windows\Shell\
copy /y C:\StartMenuLayout.xml C:\Users\RAJU\AppData\Local\Microsoft\Windows\Shell\
After exported .xml file , I clearly analysis  using notepad  , I found the  following  the shortcuts links  are also pinned  in Start menu
%ALLUSERSPROFILE%\Microsoft\Windows\Start Menu\Programs
%APPDATA%\Microsoft\Windows\Start Menu\Programs
%APPDATA%\Microsoft\Windows\Start Menu\Programs\System Tools\
I  have screenshots  of  exported and  imported . If you give permission to add those pics , I will pull requests.

Hi @RAJU2529 ,
Thank you very much for your feedback. I would like to list your suggestions with my comments before I edit the article.

Mention the default location of the Start Menu Layout. - I will add this to the article.
Specify full command for export. - The link to the detailed command is included. So as a policy, we should not duplicate. If a user wishes to understand much deeper, the Export-StartLayout specific article link explains it best.
Adding copy command. - This is a default copy command. Which I believe as a command prompt command for copying a file. I also believe a manual copy and paste will work as well. Kindly let me know if not.
Additional short cut menus on Start Menu - Kindly clarify whether you suggest this as a Windows issue related to the export command?

All the mentioned conditions are correct. You can those into article.
Point 4 is export command. It's not an Windows issues.
I think its hidden secrets.

@RAJU2529 Thank you very much for quick response. @mypil As we discussed here I see #1 of my previous comment should be in the article. But I could not find #4 mentioned in anywhere else. So shall I consider adding it? or ignore it as it is not mentioned in any guidelines as a feature of the command? #2 and #3 will not be added as there are different reference articles for them that clarifies them to a better extent.

#4 You can add as extra informations

@mypil Created PR https://github.com/MicrosoftDocs/windows-itpro-docs/pull/3821

@RAJU2529 - From our understanding, the issue has been resolved but it may take a few days for the merged content to appear in the article. If you feel it hasn't been resolved, please re-open this issue.
Thank you for your contribution to make the docs better! Much appreciated!

ok madam. you can cLose now. I keep waiting for merged content. thanks a lot

@officedocsbot close


Add JsonRpsee-socks5-proxy

Project Abstract

Please replace these instructions with a brief description of your project summarising key points (1-2 paragraphs).
If your application is a follow-up to a previous grant, please mention which one in the first line of the abstract and include a link to previous pull requests if applicable.

Grant level

[x] Level 1:  Up to $10,000, 2 approvals
[ ] Level 2:  Up to $30,000, 3 approvals
[ ] Level 3:  Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

[x] The application template has been copied and aptly renamed (project_name.md).
[x] I have read the application guidelines.
[x] Payment details have been provided (bank details via email or Polkadot (USDC & USDT) or BTC address in the application).
[x] The software delivered for this grant will be released under an open-source license specified in the application.
[x] The initial PR contains only one commit (squash and force-push if needed).
[x] The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).
[ ] I prefer the discussion of this application to take place in a private Element/Matrix channel. My username is: @_______:matrix.org (change the homeserver if you use a different one)

I have read and hereby sign the Contributor License Agreement.

recheck

Hey, @keeganquigley @Noc2, I encountered some technical problems and cannot continue. Please terminate this project.

Thanks for the update and being transparent about it. I will create the PR.

hi, @keeganquigley I cannot continue to complete this project before the deadline. btw. I have completed Websocket socks5 proxy support in my previous work, and I hope to give other teams some simple references. https://github.com/gmajor-encrypt/jsonrpsee


fix(toml): Be more forceful with underscore/dash redundancy

What does this PR try to resolve?
This is prep for removing them in the 2024 Edition and is part of rust-lang/rust#123754 and #13629
During #13783, I had considered making the 2024 edition behavior a "unused key" warning.  However, the work and code mess to pipe the data through correctly handle the two fields in all cases didn't seem worth it (and a hard error might be better to help users transition).
How should we test and review this PR?
Additional information

Odd failure I can't reproduce locally
---- global_cache_tracker::max_download_size stdout ----
running `/home/runner/work/cargo/cargo/target/debug/cargo clean gc -Zgc -v --max-download-size=30`
running `/home/runner/work/cargo/cargo/target/debug/cargo clean gc -Zgc -v --max-download-size=29`
running `/home/runner/work/cargo/cargo/target/debug/cargo clean gc -Zgc -v --max-download-size=24`
running `/home/runner/work/cargo/cargo/target/debug/cargo clean gc -Zgc -v --max-download-size=20`
running `/home/runner/work/cargo/cargo/target/debug/cargo clean gc -Zgc -v --max-download-size=1`
thread 'global_cache_tracker::max_download_size' panicked at tests/testsuite/global_cache_tracker.rs:1138:14:

test failed running `/home/runner/work/cargo/cargo/target/debug/cargo clean gc -Zgc -v --max-download-size=1`
error: Expected lines did not match (ignoring order):
0   1         Removing /home/runner/work/cargo/cargo/target/tmp/cit/t1586/home/.cargo/registry/src/example.com-a6c4a5adcb232b9a/d-1.0.0
1   4         Removing /home/runner/work/cargo/cargo/target/tmp/cit/t1586/home/.cargo/registry/cache/example.com-a6c4a5adcb232b9a/d-1.0.0.crate
2   1         Removing /home/runner/work/cargo/cargo/target/tmp/cit/t1586/home/.cargo/registry/src/example.com-a6c4a5adcb232b9a/c-1.0.0
3   3         Removing /home/runner/work/cargo/cargo/target/tmp/cit/t1586/home/.cargo/registry/cache/example.com-a6c4a5adcb232b9a/c-1.0.0.crate
4   1         Removing /home/runner/work/cargo/cargo/target/tmp/cit/t1586/home/.cargo/registry/src/example.com-a6c4a5adcb232b9a/a-1.0.0
5   2         Removing /home/runner/work/cargo/cargo/target/tmp/cit/t1586/home/.cargo/registry/cache/example.com-a6c4a5adcb232b9a/a-1.0.0.crate
6   1         Removing /home/runner/work/cargo/cargo/target/tmp/cit/t1586/home/.cargo/registry/src/example.com-a6c4a5adcb232b9a/b-1.0.0
7        -     Removed 7 files, 29B total
    7    +     Removed 8 files, 30B total
    8    +    Removing /home/runner/work/cargo/cargo/target/tmp/cit/t1586/home/.cargo/registry/cache/example.com-a6c4a5adcb232b9a/b-1.0.0.crate

@bors r+

:pushpin: Commit 034ef3c27beeba3300c516c83fd1de99b5541f5d has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit 034ef3c27beeba3300c516c83fd1de99b5541f5d with merge e3d42b6019e47c0d9708b88291687a4019d8856a...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing e3d42b6019e47c0d9708b88291687a4019d8856a to master...


Feature request: Table conversion

It'd be lovely if this program could take inserted spreadsheets to make markdown tables. Currently I'm C/V any spreadsheets into https://tabletomarkdown.com/convert-spreadsheet-to-markdown/ to convert any spreadsheets to tables.

I'll evaluate this feature when I have time.

Hi, now tables are converted to markdown too. You can install the new version with pip install pptx2md==1.1.1 and try it out.


ZF2 not including port in dsn parameter

Hi,
I am using 2.2.6, and I am not sure if this is fixed but here it goes:
In Zend\Db\Adapter\Driver\Pdo\Connection.php in method connect() the port parameter doesn't get included in the final $dns string.

Should be possible in ZF 2.2.6, because it was added in ZF 2.0.3 through PR #2622.

there is a case when $dsn is set and $pdodriver is not set, if so $port doesn't get set

there is a case when $dsn is set and $pdodriver is not set.

Can you elaborate on that? As far as I know you always need a PDO driver name. Can you show what the DSN you use looks like?

Tested your suggestion with $dsn and without $pdodriver and still works fine for me:
public function testDsnWithoutDriverNameAddsPortToDsn()
{
    $this->connection->setConnectionParameters(array(
        'dsn' => 'host=<IP_ADDRESS>;charset=utf8;dbname=foo;port=3306',
    ));
    try {
        $this->connection->connect();
    } catch (\Exception $e) {
    }
    $responseString = $this->connection->getDsn();

    $this->assertContains('host=<IP_ADDRESS>', $responseString);
    $this->assertContains('charset=utf8', $responseString);
    $this->assertContains('dbname=foo', $responseString);
    $this->assertContains('port=3306', $responseString);
}

This is what I have in my autoload/local.php
'db' => array(
        'username' => 'root',
        'password' => '',
        'driver'         => 'Pdo',
        'dsn'            => 'mysql:dbname=staging;host=<IP_ADDRESS>',
        'driver_options' => array(
            PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
        ),
        'adapters' => array(
            'db1' => array(
                'username' => 'api',
                'password' => 'pass',
                'port' => '30002',
                'driver'         => 'Pdo',
                'dsn'            => 'mysql:host=myhost.net;dbname=db1',
                'driver_options' => array(
                    PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
                ),
            ),
            'db2' => array(
                'username' => 'api',
                'password' => 'pass',
                'port' => '30002',
                'driver'         => 'Pdo',
                'dsn'            => 'mysql:dbname=db2;host=myhost.net,
                'driver_options' => array(
                    PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
                ),
            ),
        ),
    )

Ah, okay, I see where this is going wrong. You are mixing two ways of setting connection parameters: dsn and port. Zend\Db\Adapter\Pdo\Connection expects a dsn or an array of parameters.
You can fix this by adding the port to your dsn:
'dsn' => 'mysql:host=myhost.net;dbname=db1;port=30002',

Not sure this needs fixing at all, perhaps an exception should be thrown when mixing dsn and an array of parameters? Ping @Ocramius

Ping @Ocramius @weierophinney
Can you clarify if this needs a fix or if you consider it not an issue?

Have you tried connecting to a different port? I tried to include the port in the string
'dsn' => 'mysql:host=myhost.net;dbname=db1;port=30002'

but it doesn't work. I was pulling my hair out...I had to modifiy Zend/Db/Adapter/Pdo/Connection.php and add the last else case:
if (!isset($dsn) && isset($pdoDriver)) {

            $dsn = array();
            switch ($pdoDriver) {
                case 'sqlite':
                    $dsn[] = $database;
                    break;
                default:
                    if (isset($database)) {
                        $dsn[] = "dbname={$database}";
                    }
                    if (isset($hostname)) {
                        $dsn[] = "host={$hostname}";
                    }
                    if (isset($port)) {
                        $dsn[] = "port={$port}";
                    }
                    break;
            }
            $dsn = $pdoDriver . ':' . implode(';', $dsn);
        } elseif (!isset($dsn)) {
            throw new Exception\InvalidConnectionParametersException(
                'A dsn was not provided or could not be constructed from your parameters',
                $this->connectionParameters
            );
        } else {
            // there is a case when $dsn is set and $pdodriver is not set, if so port doesn't get set
            if ( $port !== null) {
                $dsn .= ';port='.$port;
            }
        }

Works great now
In the current code, as it loops through, the $dsn string gets rebuilt and leaves out the port specified in the $dsn string.

In the current code, as it loops through, the $dsn string gets rebuilt and leaves out the port specified in the $dsn string.

The problem is Zend\Db\Adapter\Pdo\Connection expects a dsn or an array of parameters. You provide both and Zend\Db\Adapter\Pdo\Connection does not support that.
Please look at my previous comment, I already gave you the solution :wink: When you apply that to your configuration you will get this:
<?php
'db' => array(
    'username' => 'root',
    'password' => '',
    'driver'         => 'Pdo',
    'dsn'            => 'mysql:dbname=staging;host=<IP_ADDRESS>',
    'driver_options' => array(
        PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
    ),
    'adapters' => array(
        'db1' => array(
            'username' => 'api',
            'password' => 'pass',
            // 'port' => '30002', remove this line
            'driver'         => 'Pdo',
            'dsn'            => 'mysql:host=myhost.net;dbname=db1;port=30002', // add it here
            'driver_options' => array(
                PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
            ),
        ),
        'db2' => array(
            'username' => 'api',
            'password' => 'pass',
            // 'port' => '30002', remove this line
            'driver'         => 'Pdo',
            'dsn'            => 'mysql:dbname=db2;host=myhost.net;port=30002', // add it here
            'driver_options' => array(
                PDO::MYSQL_ATTR_INIT_COMMAND => 'SET NAMES \'UTF8\''
            ),
        ),
    ),
)

This issue has been moved from the zendframework repository as part of the bug migration program as outlined here - http://framework.zend.com/blog/2016-04-11-issue-closures.html
New issue can be found at: https://github.com/zendframework/zend-db/issues/139


raise RuntimeError( RuntimeError: Tkinter Designer requires internet access to work.

even tho am connected through the internet

I have the same Error. Does anyone have a solution or know the cause?
Please help!
Thx!

I have the same error. Maybe it is the problem that it is the work pc with special internet settings? I would apprechiate a fix or maybe a bypass?


Train pipeline MixUp augmentation

I am trying to add the MixUp augmentation into my training pipeline (a faster rcnn model).
 File "/usr/local/lib/python3.6/dist-packages/mmdet/datasets/custom.py", line 195, in __getitem__     data = self.prepare_train_img(idx)   File "/usr/local/lib/python3.6/dist-packages/mmdet/datasets/custom.py", line 218, in prepare_train_img     return self.pipeline(results)   File "/usr/local/lib/python3.6/dist-packages/mmdet/datasets/pipelines/compose.py", line 41, in __call__     data = t(data)   File "/usr/local/lib/python3.6/dist-packages/mmdet/datasets/pipelines/transforms.py", line 2239, in __call__     results = self._mixup_transform(results)   File "/usr/local/lib/python3.6/dist-packages/mmdet/datasets/pipelines/transforms.py", line 2270, in _mixup_transform     assert 'mix_results' in results AssertionError
I was able to trace through the implementation of MixUp augmentation in pipelines/transforms.py and reached the assert statement. However, I am not understanding when is my results dict modified to ensure that the key (mix_results) can be added into the results dict for the augmentation.

adding my config file here
`model = dict(
type='FasterRCNN',
backbone=dict(
type='ResNet',
depth=50,
num_stages=4,
out_indices=(0, 1, 2, 3),
frozen_stages=1,
norm_cfg=dict(type='BN', requires_grad=True),
norm_eval=True,
style='pytorch',
init_cfg=dict(type='Pretrained', checkpoint='torchvision://resnet50')),
neck=dict(
type='FPN',
in_channels=[256, 512, 1024, 2048],
out_channels=256,
num_outs=5),
rpn_head=dict(
type='RPNHead',
in_channels=256,
feat_channels=256,
anchor_generator=dict(
type='AnchorGenerator',
scales=[8],
ratios=[0.5, 1.0, 2.0],
strides=[4, 8, 16, 32, 64]),
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0.0, 0.0, 0.0, 0.0],
target_stds=[1.0, 1.0, 1.0, 1.0]),
loss_cls=dict(
type='CrossEntropyLoss', use_sigmoid=True, loss_weight=1.0),
loss_bbox=dict(type='L1Loss', loss_weight=1.0)),
roi_head=dict(
type='StandardRoIHead',
bbox_roi_extractor=dict(
type='SingleRoIExtractor',
roi_layer=dict(type='RoIAlign', output_size=7, sampling_ratio=0),
out_channels=256,
featmap_strides=[4, 8, 16, 32]),
bbox_head=dict(
type='Shared2FCBBoxHead',
in_channels=256,
fc_out_channels=1024,
roi_feat_size=7,
num_classes=1,
bbox_coder=dict(
type='DeltaXYWHBBoxCoder',
target_means=[0.0, 0.0, 0.0, 0.0],
target_stds=[0.1, 0.1, 0.2, 0.2]),
reg_class_agnostic=False,
loss_cls=dict(
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
loss_bbox=dict(type='L1Loss', loss_weight=1.0))),
train_cfg=dict(
rpn=dict(
assigner=dict(
type='MaxIoUAssigner',
pos_iou_thr=0.7,
neg_iou_thr=0.3,
min_pos_iou=0.3,
match_low_quality=True,
ignore_iof_thr=-1),
sampler=dict(
type='RandomSampler',
num=256,
pos_fraction=0.5,
neg_pos_ub=-1,
add_gt_as_proposals=False),
allowed_border=-1,
pos_weight=-1,
debug=False),
rpn_proposal=dict(
nms_pre=2000,
max_per_img=1000,
nms=dict(type='nms', iou_threshold=0.7),
min_bbox_size=0),
rcnn=dict(
assigner=dict(
type='MaxIoUAssigner',
pos_iou_thr=0.5,
neg_iou_thr=0.5,
min_pos_iou=0.5,
match_low_quality=False,
ignore_iof_thr=-1),
sampler=dict(
type='RandomSampler',
num=512,
pos_fraction=0.25,
neg_pos_ub=-1,
add_gt_as_proposals=True),
pos_weight=-1,
debug=False)),
test_cfg=dict(
rpn=dict(
nms_pre=1000,
max_per_img=1000,
nms=dict(type='nms', iou_threshold=0.7),
min_bbox_size=0),
rcnn=dict(
score_thr=0.05,
nms=dict(type='nms', iou_threshold=0.5),
max_per_img=100)))
img_norm_cfg = dict(
mean=[123.675, 116.28, 103.53], std=[58.395, 57.12, 57.375], to_rgb=True)
train_pipeline = [
dict(type='LoadImageFromFile'),
dict(type='LoadAnnotations', with_bbox=True),
dict(
type='Resize',
img_scale=[(1333, 640), (1333, 672), (1333, 704), (1333, 736),
(1333, 768), (1333, 800)],
multiscale_mode='value',
keep_ratio=True),
dict(type='RandomFlip', flip_ratio=0.5),
dict(type='RandomAffine', max_rotate_degree=180, max_translate_ratio=0.25),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='MixUp'),
dict(type='DefaultFormatBundle'),
dict(type='Collect', keys=['img', 'gt_bboxes', 'gt_labels'])
]
test_pipeline = [
dict(type='LoadImageFromFile'),
dict(
type='MultiScaleFlipAug',
img_scale=(1333, 800),
flip=False,
transforms=[
dict(type='Resize', keep_ratio=True),
dict(type='RandomFlip'),
dict(
type='Normalize',
mean=[123.675, 116.28, 103.53],
std=[58.395, 57.12, 57.375],
to_rgb=True),
dict(type='Pad', size_divisor=32),
dict(type='ImageToTensor', keys=['img']),
dict(type='Collect', keys=['img'])
])
]
data = dict(
samples_per_gpu=2,
workers_per_gpu=0,
train=dict(
type='LogoDataset',
ann_file='litw/train/train.json',
img_prefix='images/train',
pipeline=train_pipeline,
data_root='/workspace/jupyter_notebooks/datasets/mmdet'),
val=dict(
type='LogoDataset',
ann_file='litw/val/val.json',
img_prefix='images/val',
pipeline=test_pipeline,
data_root='/workspace/jupyter_notebooks/datasets/mmdet'),
test=dict(
type='CocoLogoDataset',
ann_file='kv_val.json',
img_prefix='images',
pipeline=test_pipeline,
data_root='/workspace/jupyter_notebooks/datasets/mmdet/kv_val'))
evaluation = dict(interval=1, metric='mAP')
optimizer = dict(type='SGD', lr=0.0025, momentum=0.9, weight_decay=0.0001)
optimizer_config = dict(grad_clip=dict(max_norm=35, norm_type=2))
lr_config = dict(
policy='step',
warmup='linear',
warmup_iters=500,
warmup_ratio=0.001,
step=[8, 11])
runner = dict(type='EpochBasedRunner', max_epochs=5)
checkpoint_config = dict(interval=1)
log_config = dict(interval=500, hooks=[dict(type='TextLoggerHook')])
custom_hooks = [dict(type='NumClassCheckHook')]
dist_params = dict(backend='nccl')
log_level = 'INFO'
load_from = 'checkpoints/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth'
resume_from = None
workflow = [('train', 1)]
work_dir = 'models/frcnn_aug'
total_epochs = 5
seed = 0
gpu_ids = range(0, 1)
`

Okay, we have to wrap our dataset with MultiImageMixDataset to wrap around your dataset. Can follow the config found in yolox

Did you solve it ? I added it by referencing with the mode  yolox cfg, but also happen this problem, can you show me how to fix it?

I did so as the mode yolox cfg, it now can run, but not being completely, because the training is terminated sometimes.
@.***
From: Lim Jun Hup
Date: 2021-10-13 15:47
To: open-mmlab/mmdetection
CC: y78h11b09; Mention
Subject: Re: [open-mmlab/mmdetection] Train pipeline MixUp augmentation (#6122)
Did you solve it ? I added it by referencing with the mode yolox cfg, but also happen this problem, can you show me how to fix it?
@y78h11b09 sorry for the late response, but the example code can be found in the yolox config.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
Triage notifications on the go with GitHub Mobile for iOS or Android.

How is your result of Faster RCNN on Coco dataset? I got poor results using mixup or mosiac data augmentation methods.


Add pulsating heart

The pulsating heart of the loved frog should motivate us!

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 4 (0.0%)  changed or added relevant lines in 1 file are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.008%) to 0.866%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

burpeefrog/main.py
0
4
0.0%

Files with Coverage Reduction
New Missed Lines
%

burpeefrog/main.py
1
0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.008%

Covered Lines:
4

Relevant Lines:
462

💛  - Coveralls