Bug? PEBCAK? Unable to initialize __nuint_1 to user-defined length

Actual behavior
Full disclosure: I am probably missing something or do not know how to do something (although, I have spent the day reading and researching and testing).
When calling an API and passing a structure which contains a variable array, CsWin32 generates a struct that has a member of type __nuint_1. However, I see no way to set the __nuint_1's SpanLength so that it allocates a specific number of elements.
Expected behavior
It seems that __nuint_1 should have a way to initialize it to have a user-specified number of elements, since we have to pass the size of the memory block to the function.
Repro steps

NativeMethods.txt content:

QueryInformationJobObject
JOBOBJECT_BASIC_PROCESS_ID_LIST

Generated:
	internal partial struct JOBOBJECT_BASIC_PROCESS_ID_LIST
	{
		internal uint NumberOfAssignedProcesses;
		internal uint NumberOfProcessIdsInList;
		internal winmdroot.__nuint_1 ProcessIdList;
	}

	internal partial struct __nuint_1
	{
		private const int SpanLength = 1;

		/// <summary>The length of the inline array.</summary>
		internal readonly int Length => SpanLength;
		internal nuint _0;

		/// <summary>
		/// Gets this inline array as a span.
		/// </summary>
		/// <remarks>
		/// ⚠ Important ⚠: When this struct is on the stack, do not let the returned span outlive the stack frame that defines it.
		/// </remarks>
		[UnscopedRef]
		internal unsafe Span<nuint> AsSpan() => MemoryMarshal.CreateSpan(ref _0, SpanLength);

		/// <summary>
		/// Gets this inline array as a span.
		/// </summary>
		/// <remarks>
		/// ⚠ Important ⚠: When this struct is on the stack, do not let the returned span outlive the stack frame that defines it.
		/// </remarks>
		[UnscopedRef]
		internal unsafe readonly ReadOnlySpan<nuint> AsReadOnlySpan() => MemoryMarshal.CreateReadOnlySpan(ref Unsafe.AsRef(_0), SpanLength);
		public static implicit operator __nuint_1(ReadOnlySpan<nuint> value)
		{
			Unsafe.SkipInit(out __nuint_1 result);
			value.CopyTo(result.AsSpan());
			int initLength = value.Length;
			result.AsSpan().Slice(initLength, SpanLength - initLength).Clear();
			return result;
		}
	}

Any of your own code that should be shared?

	JOBOBJECT_BASIC_PROCESS_ID_LIST idList = new();
	uint nData = (uint)(Marshal.SizeOf(idList) + Marshal.SizeOf(idList.ProcessIdList) * 100);   // We want to allocate 100 elements
	PInvoke.QueryInformationJobObject(hJob, JOBOBJECTINFOCLASS.JobObjectBasicProcessIdList, &idList, nData, &returnLength);

Context

CsWin32 version: 0.2.188-beta
Win32Metadata version (if explicitly set by project):
Target Framework: .NET 7
LangVersion (if explicitly set by project): latest (C# 11)

The problem is CsWin32 doesn't recognize this as a variable length array, so it just uses a fixed 1-length array.
See #387, which tracks improving the usability of this case.
As a workaround, assuming the struct is initialized elsewhere, you can access the full list like this:
JOBOBJECT_BASIC_PROCESS_ID_LIST list;
unsafe
{
    Span<nuint> pids = new(&list.ProcessIdList._0, (int)list.NumberOfProcessIdsInList);
    for (int i = 0; i < pids.Length; i++)
    {
        Console.WriteLine(pids[i]);
    }
}

But if you need to initialize the list, you'll have to allocate enough memory for the struct, and then pin the struct at that location yourself. I can provide sample code for that if you need.

Yes, QueryInformationJobObject requires the caller to allocate the memory, and I have been unsuccessful at coercing a block of memory to be a JOBOBJECT_BASIC_PROCESS_ID_LIST. If you could provide that sample code, I would appreciate it.

JOBOBJECT_BASIC_PROCESS_ID_LIST idList = *pidList;
var ids = idList.ProcessIdList.AsReadOnlySpan();

This is broken for two reasons: AsReadOnlySpan() will only give you the first item. And secondly, because your *pidList expression will copy the struct from native memory onto your stack, without all the elements after the first one.
I'm preparing a sample of what I believe should work.

using Windows.Win32;
using Windows.Win32.Foundation;
using Windows.Win32.System.JobObjects;

unsafe
{
    HANDLE hJob = default;
    const int MaxProcessCount = 4096; // TODO: pick a good number here
    int MemorySize = sizeof(JOBOBJECT_BASIC_PROCESS_ID_LIST) + MaxProcessCount * sizeof(nuint);
    fixed (byte* pBytes = new byte[MemorySize])
    {
        JOBOBJECT_BASIC_PROCESS_ID_LIST* pList = (JOBOBJECT_BASIC_PROCESS_ID_LIST*)pBytes;

        uint returnLength = 0;
        PInvoke.QueryInformationJobObject(hJob, JOBOBJECTINFOCLASS.JobObjectBasicProcessIdList, pList, (uint)MemorySize, &returnLength);

        Span<nuint> pids = new(&pList->ProcessIdList, (int)pList->NumberOfProcessIdsInList);
        for (int i = 0; i < pids.Length; i++)
        {
            Console.WriteLine(pids[i]);
        }
    }
}

I'm using memory allocated from the GC heap so that you don't have to worry about a try/finally block to ensure you don't leak unmanaged memory.

Thank you very much, Andrew. I'll give that a go.


mdbx.lib built with MSVC LTO prevents linking with lld

Is it possible to specify an option to cmake, so that LTO will be disabled?
I'm building with
mkdir build
cd build
cmake -G "Visual Studio 16 2019" -A x64 -D CMAKE_CONFIGURATION_TYPES="Debug;Release;RelWithDebInfo" -D MDBX_AVOID_CRT:BOOL=ON -D MDBX_BUILD_SHARED_LIBRARY:BOOL=OFF ..
cmake --build . --config RelWithDebInfo

The workaround I found is to replace in libmdbx/cmake/compiler.cmake the set(MSVC_LTO_AVAILABLE TRUE) with set(MSVC_LTO_AVAILABLE FALSE) to prevent the use of /LTCG, which messes with lld linker.

Basically LTO should be controllable by the INTERPROCEDURAL_OPTIMIZATION (i.e. be the well known CMake's option).
If it will works as you need, then close this issue.
Otherwise clarify/actualize the trouble.

Indeed, adding -D INTERPROCEDURAL_OPTIMIZATION:BOOL=FALSE worked as needed. Thanks


t2 cli can't find my tessels

From @Frijol on October 22, 2015 4:3
Been having this issue where sometimes t2 commands just can't find Tessels. Typical output:
➜  t2 provision
INFO Looking for your Tessel...
WARN No Tessels Found.

Copied from original issue: tessel/cli#188

Tried debug scripts, hit timeouts:
➜ DEBUG=discovery,discovery:* t2 list
INFO Searching for nearby Tessels...
  discovery Will scan for LAN devices +0ms
  discovery Will scan for USB devices +1ms
  discovery Timeout hit! Waiting for pending to finish... +5s
  discovery Done! Stopping. +2ms
WARN No Tessels Found.
➜  DEBUG=discovery,discovery:* t2 list -t 30
INFO Searching for nearby Tessels...
  discovery Will scan for LAN devices +0ms
  discovery Will scan for USB devices +1ms
  discovery Timeout hit! Waiting for pending to finish... +30s
  discovery Done! Stopping. +2ms
WARN No Tessels Found.

It's showing up on serial, that's good – this autocompletes:
➜ ls /dev/tty.usbmodem1412

From @johnnyman727 on October 22, 2015 4:5
@Frijol can you dterm in and run ps? We're checking to see if the usbdaemon is running.

Unplugged/replugged Tessel & opened new terminal window. Dtermed in
➜  ~  v2
0] random: nonblocking pool is initialized

BusyBox v1.23.2 (2015-06-23 05:35:53 UTC) built-in shell (ash)

Tessel 2  /  Built on OpenWrt
root@Tessel-02A3C66A13C7:/#

froze:
➜  alias v2="dterm /dev/tty.usbmodem1412"
➜  v2

dterm> ^C
^C^C^C^C^C^]^]^]

exit
^]^C

(also exited dterm, still experiencing the problem):
➜  ~  t2 list
INFO Searching for nearby Tessels...
WARN No Tessels Found.

Results of ps - what are we looking for?
root@Tessel-02A3C66A13C7:/# ps
  PID USER       VSZ STAT COMMAND
    1 root      1436 S    /sbin/procd
    2 root         0 SW   [kthreadd]
    3 root         0 SW   [ksoftirqd/0]
    4 root         0 SW   [kworker/0:0]
    5 root         0 SW<  [kworker/0:0H]
    6 root         0 SW   [kworker/u2:0]
    7 root         0 SW<  [khelper]
    8 root         0 SW   [kworker/u2:1]
   69 root         0 SW<  [writeback]
   71 root         0 SW<  [bioset]
   74 root         0 SW<  [kblockd]
   76 root         0 SW   [kswapd0]
   77 root         0 SW   [kworker/0:1]
   78 root         0 SW   [fsnotify_mark]
   80 root         0 SW   [spi32766]
  223 root         0 SW<  [deferwq]
  270 root         0 SW   [kworker/u2:2]
  282 root         0 SW   [kworker/0:2]
  306 root         0 SWN  [jffs2_gcd_mtd6]
  371 root       900 S    /sbin/ubusd
  409 root      1488 S    /bin/ash --login
  779 root         0 SW<  [ipv6_addrconf]
  865 root         0 SW<  [krfcommd]
  915 root         0 SW<  [cfg80211]
 1023 root      1056 S    /sbin/logd -S 16
 1032 root      1864 S    /sbin/rpcd
 1066 root      1572 S    /sbin/netifd
 1100 root      1192 S    /usr/sbin/odhcpd
 1138 root      1148 S    /usr/sbin/dropbear -F -P /var/run/dropbear.1.pid -s
 1147 root      1732 S    /usr/sbin/dbus-daemon --system
 1157 nobody    1324 S    /usr/sbin/mdnsd -debug
 1166 root       772 S    spid /dev/spidev32766.1 2 1 /var/run/tessel
 1209 root      1484 S    udhcpc -p /var/run/udhcpc-eth0.pid -s /lib/netifd/dh
 1271 root      1140 S    /usr/bin/mDNSResponder -b -f /etc/mDNSResponder.conf
 1281 root       904 S    dns-sd -R Tessel-02A3C66A13C7 _tessel._tcp local 22
 1291 root      3436 S    /usr/bin/bluetoothd -n
 1314 nobody     980 S    /usr/sbin/dnsmasq -C /var/etc/dnsmasq.conf -k -x /va
 1344 root      1484 S    /usr/sbin/ntpd -n -S /usr/sbin/ntpd-hotplug -p 0.ope
 1356 root       780 S    usbexecd /var/run/tessel/usb
 1361 root      1484 R    ps

From @johnnyman727 on October 22, 2015 4:13
So we can see the usb daemon is running in this line:
1356 root       780 S    usbexecd /var/run/tessel/usb

To test if the usb daemon might be in a bad state, try running /etc/init.d/usbexecd restart. That will restart the usb daemon. Then try running DEBUG=discovery,discovery:* t2 list again (from outside dterm).

ugh froze starting dterm again.
New terminal window. Tried starting dterm from there without rebooting Tessel. Froze again.
New terminal window. Rebooted Tessel. This time it worked - so something on Tessel's USB comms isn't closing right.

Can you give https://github.com/tessel/t2-cli/pull/386 a try?

Did the usbexecd restart within dterm, then ran the DEBUG t2 list again. It found my Tessel!
➜  ~  DEBUG=discovery,discovery:* t2 list
INFO Searching for nearby Tessels...
  discovery Will scan for LAN devices +0ms
  discovery Will scan for USB devices +2ms
  discovery:usb Device found. +5ms
  discovery Connection opened: +11ms undefined
  discovery Fetching name: +0ms undefined
  discovery:lan Device found: +453ms Hangy._tessel._tcp.local
  discovery:lan Device open error: +3s Hangy.local
  discovery Connection opened: +2ms Hangy.local
	Hangy	LAN	(USB connect and run `t2 provision` to authorize)
  discovery Timeout hit! Waiting for pending to finish... +2s
  discovery Done! Stopping. +3ms

Let me try to run some commands before I check out #386

Welp, that was it. It found that usb Device (notably never printed it). Then on a second run of that command, we don't even get that:
➜  ~  DEBUG=discovery,discovery:* t2 list
INFO Searching for nearby Tessels...
  discovery Will scan for LAN devices +0ms
  discovery Will scan for USB devices +2ms
  discovery:lan Device found: +136ms Hangy._tessel._tcp.local
  discovery:lan Device open error: +2s Hangy.local
  discovery Connection opened: +4ms Hangy.local
	Hangy	LAN	(USB connect and run `t2 provision` to authorize)
  discovery Timeout hit! Waiting for pending to finish... +3s
  discovery Done! Stopping. +3ms

Checking out #386 now

On #386, looks the same:
➜  ~  DEBUG=discovery,discovery:* t2 list
INFO Searching for nearby Tessels...
  discovery Will scan for LAN devices +0ms
  discovery Will scan for USB devices +2ms
  discovery Timeout hit! Waiting for pending to finish... +5s
  discovery Done! Stopping. +2ms
WARN No Tessels Found.

/etc/init.d/usbexecd restart is working, I just got myself on wifi & provisioned so now I'm not dependent on the USB comms... anything else I should try to help pinpoint the issue, thought?

Managed an update (LEDs currently flashing a reboot) but got this angry readout:
➜  ~  t2 update
INFO Looking for your Tessel...
WARN -1 Tessel devices connected by USB may still be booting.
INFO Connected to Tessel-02A3FD15D69B over LAN
WARN Could not find firmware version on Tessel-02A3FD15D69B
WARN Forcefully updating...
INFO Updating Tessel-02A3FD15D69B to latest version (0.0.4)...
INFO Beginning update download. This could take a couple minutes..
  downloading [====================] 100% 0.0s remaining
INFO Download complete!
INFO Updating OpenWRT (1/2)
INFO Transferring image of size 19.92 MB
INFO Starting OpenWRT update. Please do not remove power from Tessel.
INFO Updating firwmare (2/2)
INFO Firmware update complete!
INFO Tessel is rebooting. Please wait about 30 seconds before using Tessel.
INFO Force updated Tessel-02A3FD15D69B to version 0.0.4
Unhandled rejection Error: LIBUSB_ERROR_NO_DEVICE
    at Error (native)
    at InEndpoint.Endpoint.stopPoll (/Users/frijol/t2-cli/node_modules/usb/usb.js:259:25)
    at USB.Connection._close (/Users/frijol/t2-cli/lib/usb_connection.js:181:13)
    at tryCatcher (/Users/frijol/t2-cli/node_modules/bluebird/js/main/util.js:26:23)
    at ret (eval at <anonymous> (/Users/frijol/t2-cli/node_modules/bluebird/js/main/promisify.js:163:12), <anonymous>:11:39)
    at /Users/frijol/t2-cli/lib/usb_connection.js:359:51
    at tryCatcher (/Users/frijol/t2-cli/node_modules/bluebird/js/main/util.js:26:23)
    at MappingPromiseArray._promiseFulfilled (/Users/frijol/t2-cli/node_modules/bluebird/js/main/map.js:56:38)
    at MappingPromiseArray.init [as _init$] (/Users/frijol/t2-cli/node_modules/bluebird/js/main/promise_array.js:92:18)
    at MappingPromiseArray.init (/Users/frijol/t2-cli/node_modules/bluebird/js/main/map.js:29:23)
    at Async._drainQueue (/Users/frijol/t2-cli/node_modules/bluebird/js/main/async.js:128:12)
    at Async._drainQueues (/Users/frijol/t2-cli/node_modules/bluebird/js/main/async.js:133:10)
    at Immediate.Async.drainQueues [as _onImmediate] (/Users/frijol/t2-cli/node_modules/bluebird/js/main/async.js:15:14)
    at processImmediate [as _immediateCallback] (timers.js:368:17)

Interestingly, it can still find it over USB.
➜  ~  t2 list
INFO Searching for nearby Tessels...
	Tessel-02A3FD15D69B	USB

...and even weirder, now that I think about it, is that it didn't reconnect to Wifi automatically.

Yes, I've seen that issue where sometimes an OpenWRT update will wipe the wireless credentials. Just made an open issue: https://github.com/tessel/t2-cli/issues/391

Trying another T2 that needed updating, this time not able to fix it
➜  ~  t2 provision
INFO Looking for your Tessel...
WARN 1 Tessel device connected by USB may still be booting.
WARN No Tessels Found.
➜  ~  v2

root@Tessel-02A3C66A13C7:/# /etc/init.d/usbexecd restart
root@Tessel-02A3C66A13C7:/#
dterm> ^C

➜  ~  t2 provision
INFO Looking for your Tessel...
WARN 1 Tessel device connected by USB may still be booting.
WARN No Tessels Found.

Using a different Tessel (still on #386) getting this issue:
➜  ~  t2 wifi
INFO Looking for your Tessel...
WARN 1 Tessel device connected by USB may still be booting.
WARN No Authorized Tessels Found.
events.js:141
      throw er; // Unhandled 'error' event
            ^
Error: Connection was already closed...
    at USB.Connection._write (/Users/frijol/t2-cli/lib/usb_connection.js:70:14)
    at doWrite (_stream_writable.js:291:12)
    at clearBuffer (_stream_writable.js:401:7)
    at onwrite (_stream_writable.js:336:7)
    at OutEndpoint.WritableState.onwrite (_stream_writable.js:88:5)
    at Transfer.callback (/Users/frijol/t2-cli/node_modules/usb/usb.js:343:14)

Not sure if dup or no but I'm having similar issues:
~/Work/tessel $ t2 version
INFO Looking for your Tessel...
INFO Connected to Tessel-02A30D3B52A3 over USB
events.js:85
      throw er; // Unhandled 'error' event
            ^
Error: Error upon remote process exit: 1
    at USBProcess.<anonymous> (/Users/jbergstroem/.nvm/versions/node/v0.12.7/lib/node_modules/t2-cli/lib/usb/usb_process.js:49:26)
    at USBProcess.g (events.js:199:16)
    at USBProcess.emit (events.js:107:17)
    at Parser.<anonymous> (/Users/jbergstroem/.nvm/versions/node/v0.12.7/lib/node_modules/t2-cli/lib/usb/usb_daemon.js:121:14)
    at Parser.emit (events.js:107:17)
    at Parser._write (/Users/jbergstroem/.nvm/versions/node/v0.12.7/lib/node_modules/t2-cli/node_modules/usb-daemon-parser/index.js:152:16)
    at doWrite (_stream_writable.js:301:12)
    at writeOrBuffer (_stream_writable.js:288:5)
    at Parser.Writable.write (_stream_writable.js:217:11)
    at ondata (_stream_readable.js:540:20)
    at emit (events.js:107:17)
    at readableAddChunk (_stream_readable.js:163:16)
    at Readable.push (_stream_readable.js:126:10)
    at InEndpoint.emit (events.js:107:17)
    at Transfer.transferDone (/Users/jbergstroem/.nvm/versions/node/v0.12.7/lib/node_modules/t2-cli/node_modules/usb/usb.js:295:9)

~/Work/tessel $ system_profiler SPUSBDataType
USB:

    USB 3.0 Bus:

      Host Controller Driver: AppleUSBXHCIWPT
      PCI Device ID: 0x9cb1
      PCI Revision ID: 0x0003
      PCI Vendor ID: 0x8086

        Bluetooth USB Host Controller:
        <snip for relevance>

        Tessel 2:

          Product ID: 0x7551
          Vendor ID: 0x1209
          Version: 1.10
          Serial Number: <changed if sensitive for whatever reason>
          Speed: Up to 12 Mb/sec
          Manufacturer: Technical Machine
          Location ID: 0x14100000 / 11
          Current Available (mA): 1000
          Current Required (mA): 500
          Extra Operating Current (mA): 0

..after the failed connection attempt it cannot be identified.

FWIW, updating to 0.0.5 seems to have fixed above connectivity issues for me.

You're getting that every time? That's usually what it looks like if you disconnect USB mid-connection

@jbergstroem could you try rebooting your T2 and trying out these debugging steps? I'd love to know 1. How many USB Daemon processes you see and 2. if restarting them before running the t2 version command help.

@johnnyman727 restarting always helped. I've updated the firmware and it seems to a smooth ride from here on. Will have a closer look tomorrow.

This should be fixed with #482 and #486.


I can't get select2 dependent fields work

I am trying to implement dependent select fields as shown in documents .
My models are shown here:

class VehicleBrand(BaseModel):
"""Brands for types of vehicles"""

name = models.CharField(_('Name'), max_length=50)

def __str__(self):
    return self.name

class VehicleModel(BaseModel):
"""vehicle model"""

brand = models.ForeignKey(
    'awesomeinventory.VehicleBrand',
    verbose_name=_('Brand of the vehicle'),
    on_delete=models.CASCADE,
    related_name='models',
)
name = models.CharField(_("Vehicle model name"), max_length=50)

def __str__(self):
    return self.name

My forms.py looks like:

class MarketTrackForm(forms.Form):
brand = forms.ModelChoiceField(
    queryset=VehicleBrand.objects.all(),
    to_field_name='name',
    label=u'Brand',
    widget=ModelSelect2Widget(
        model=VehicleBrand,
        search_fields=['name__icontains'],
        attrs={'style': 'width: 100%', 'data-minimum-input-length': 0},
    ),
)
model = forms.ModelChoiceField(
    queryset=VehicleModel.objects.all(),
    to_field_name='name',
    label=u'Model',
    widget=ModelSelect2Widget(
        model=VehicleModel,
        search_fields=['name__icontains'],
        dependent_fields={'brand': 'brand'},
        max_results=500,
        attrs={
            'style': 'width: 100%',
            'data-minimum-input-length': 0,
        },
    ),
)

my urls.py

urlpatterns += i18n_patterns(
...
path('select2/', include('django_select2.urls')),
...
)

and my settings looks like

...
    CACHES = {
    "default": {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://<IP_ADDRESS>:6379/1",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        },
    },
    'select2': {
        "BACKEND": "django_redis.cache.RedisCache",
        "LOCATION": "redis://<IP_ADDRESS>:6379/2",
        "OPTIONS": {
            "CLIENT_CLASS": "django_redis.client.DefaultClient",
        },
    },
}

# Set the cache backend to select2
SELECT2_CACHE_BACKEND = 'select2'
...

When I remove dependent fields everything works. Also I tried to copy exact example shown in documentation (country and city models) and even that didn't work. when I add dependent field, it just starts to say no results found(after selecting dependent field), If I remove dependent field from form it works. So I thought it could be about settings or caching. But couldnt figure it out. Any tips would help, thank you.
---update---
When I remove

kwargs.update(
    {
        f"{model_field_name}__in": filter(
            None, self.request.GET.get(f"{form_field_name}[]", "").split(",")
        )
        for form_field_name, model_field_name in self.widget.dependent_fields.items()
    }
)

This code from select2.views.AutoResponseView it just started to work. I dont want to mess with it tho. How could I fix it without removing this pile of code and what even this pile does? anyone?

changing ModelSelect2Widget to ModelSelect2MultipleWidget solved the problem.

There also were too little bugs in the latest 7.7.0 release that was fixed in 7.7.1


How to configure .backup ?

Overview:
Documentation missing about .backup settings. Could you provider some ?

There are no settings for backup, as that is something that the extension uses for internal purposes only.  Where is a list option that when you click it, temporarily reveals all items you had hidden, and when you click that same icon again, restores your previous hidden items ( as well as which ones you had check, unchecked ).
That backup setting is just what your previous settings were before clicking the show all icon, so when you click it again, we can restore your old settings... otherwise it would be impossible to restore if VS Code did not have some other solution to track what your old settings used to be.

Could you at least make it so it's not set to null every time and instead make it something like {}?
It will just show a warning all the time because of the null value:

null is a 100% valid JSON ( see spec )
This is a system JSON file, so you will only ever even see this notice if you have the system file open.  Also, null is the correct value here.  It is not an object at this point, it is null.  It adds complexity to define this as an empty object and check if that object is empty, when it is in fact, null.
This is an issue with how VS Code parses JSON, but it is 100% correct according to the actual JSON spec:

JSON can represent four primitive types (strings, numbers, booleans, and null) and two structured types (objects and arrays).

Idea:  Mention in the description that it's for extension internal use only so we can ignore it if we stumble on it in the settings dialog?

@TomGault That's not a bad idea, I had only actually had one person even mention that backup specifically that did not know what it was for, seemed like the only other issue with it was that it was null vs {} ( even though null is valid JSON, the linter just did not like it ).  I can get that note added for the next release.

Idea #2:  I was browsing the VS Code extension API docs and found this:
https://code.visualstudio.com/api/extension-capabilities/common-capabilities#data-storage
If you stored backup in [ExtensionContext.workspaceState](https://code.visualstudio.com/api/references/vscode-api#ExtensionContext.workspaceState) then it wouldn't even show up in the Settings dialog, so there'd be nothing to have to describe to users!  🎉


Demo feature-flagging entry points as per cosmwasm-plus

Scratching my own itch here. This is how to allow a feature ("library") to disable the wasm entry_point generation. We use this heavily in cosmwasm-plus to allow one contract to import another.
Go to contracts/queue and run:
cargo wasm && cargo test and it will pass
cargo wasm --features=library && cargo test and it will fail one test as there is no migrate entry point
I am happy for a cleaner solution, but this works for now. Let's keep some approach documented in the code.

I am happy for a cleaner solution, but this works for now. Let's keep some approach documented in the code.

I think the library feature does the job for now, but is not a great solution in general. I see two problems here:

In Rust, all features I have seen so far are positive and add functionality. This "library" adds nothing and instead supresses functionality. This makes it hard to reason about features when activating multiple.
I don't think a top level contract should expose functionality for other contracts. If two contracts share some functionality, this is best organized in a shared dependency.

It is good the new entry point system works well with the current cosmwasm-plus code for a smooth migration. However, I'd not optimize for this kind of flag and hesitate to document is as best practice.

I am happy for a cleaner solution, but this works for now. Let's keep some approach documented in the code.

I think the library feature does the job for now, but is not a great solution in general. I see two problems here:

In Rust, all features I have seen so far are positive and add functionality. This "library" adds nothing and instead supresses functionality. This makes it hard to reason about features when activating multiple.
I don't think a top level contract should expose functionality for other contracts. If two contracts share some functionality, this is best organized in a shared dependency.

It is good the new entry point system works well with the current cosmwasm-plus code for a smooth migration. However, I'd not optimize for this kind of flag and hesitate to document is as best practice.

I don't think a top level contract should expose functionality for other contracts. If two contracts share some functionality, this is best organized in a shared dependency.

This is how most other smart contract languages work - "extending" other contracts to add their functionality. I have been pulling out some logic into shared dependencies, to use more like mixins, especially for "utility" items, like managing a contract "admin". However, there is a lot to be said for supporting this approach in some way or another, as this is what most contract developers will want to use. And if the "to be extended" contract if not yours, it is quite hard to organize these shared dependencies.
Let's talk about this later, but I think it is an essential point that we get a shared understanding of contract development.

I don't think a top level contract should expose functionality for other contracts. If two contracts share some functionality, this is best organized in a shared dependency.

This is how most other smart contract languages work - "extending" other contracts to add their functionality. I have been pulling out some logic into shared dependencies, to use more like mixins, especially for "utility" items, like managing a contract "admin". However, there is a lot to be said for supporting this approach in some way or another, as this is what most contract developers will want to use. And if the "to be extended" contract if not yours, it is quite hard to organize these shared dependencies.
Let's talk about this later, but I think it is an essential point that we get a shared understanding of contract development.

However, there is a lot to be said for supporting this approach in some way or another, as this is what most contract developers will want to use. And if the "to be extended" contract if not yours, it is quite hard to organize these shared dependencies.
Let's talk about this later, but I think it is an essential point that we get a shared understanding of contract development best practices.

Yeah, happy to do that. I think it is important to balance well between Rust best practice and what we want from it. And also decide it interfaces should work cross-language or not.
Anyways, this demo flag here does not hurt and demos the same as https://github.com/CosmWasm/cosmwasm-plus/pull/216

However, there is a lot to be said for supporting this approach in some way or another, as this is what most contract developers will want to use. And if the "to be extended" contract if not yours, it is quite hard to organize these shared dependencies.
Let's talk about this later, but I think it is an essential point that we get a shared understanding of contract development best practices.

Yeah, happy to do that. I think it is important to balance well between Rust best practice and what we want from it. And also decide it interfaces should work cross-language or not.
Anyways, this demo flag here does not hurt and demos the same as https://github.com/CosmWasm/cosmwasm-plus/pull/216


options(java.parameters = "-Xmx5g")  has to be moved above all library imports

Currently, options(java.parameters = "-Xmx5g")  is below library(IntegratedLearner). I was trying to run the vignette from a Podman (might also be a problem with Docker) container, and bartMachine would not load with more that ~500m of memory. After I moved this option setting before any library import, it loads with the set amount of memory.
Containerfile for your reference
# Fetch base image for R version 4.4.0
FROM rocker/rstudio:4.4.0

# Install Java
RUN apt-get update \
   && apt-get install -y --no-install-recommends \
   openjdk-21-jdk

# Let R find Java files
RUN R CMD javareconf

# Install CRAN packages
RUN R -e 'install.packages(c("bartMachine", "caret", "cowplot", "devtools", "mcmcplots", "rJava", "tidyverse"))'

# Install IntegratedLearner
RUN R -e 'devtools::install_github("himelmallick/IntegratedLearner")'

# Create directory for IntegratedLearner
WORKDIR /root/integratedlearner

# Internal port for RStudio server is 8787
EXPOSE 8787

P.S. I may also create a pull request for this Containerfile.

Hi @artur-sannikov, please feel free to create a pull request for this if you’d like. I’m happy to review and approve it. For now, I’m closing this issue. Thanks for the fix!


nc not in path

in the ftppush.sh script I've added: PATH=$PATH:/tmp/sd/yi-hack/usr/bin at line 8 because nc was not in PATH and no upload was done
The tree replication doens't works, it fails with "ftpput: unexpected server response to STOR: 550 " in log
In the ftp server log I Can't see any mkdir operation... I'll investigate

ftppush.sh already contains /tmp/sd/yi-hack/usr/bin in the PATH.
See line 18: https://github.com/roleoroleo/yi-hack-Allwinner-v2/blob/master/src/static/static/yi-hack/script/ftppush.sh

ftppush.sh already contains /tmp/sd/yi-hack/usr/bin in the PATH.
See line 18: https://github.com/roleoroleo/yi-hack-Allwinner-v2/blob/master/src/static/static/yi-hack/script/ftppush.sh


User authentication get it to work on angular-cli: 1.0.0-beta.15

get the authentication to after i write  app/app.module.ts like this...
`import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { HttpModule } from '@angular/http';
import {FIREBASE_PROVIDERS,
defaultFirebase,
AngularFire,
AuthMethods,
AuthProviders,
firebaseAuthConfig,   AngularFireModule
} from 'angularfire2';
import { AppComponent } from './app.component';
export const myFirebaseConfig = {
apiKey: "key",
authDomain: "key",
databaseURL: "key",
storageBucket: "key",
}
export const myFirebaseAuthConfig = {
provider: AuthProviders.Google,
method: AuthMethods.Redirect
}
@NgModule({
declarations: [
AppComponent
],
imports: [
BrowserModule,
AngularFireModule.initializeApp(myFirebaseConfig, myFirebaseAuthConfig),
FormsModule,
HttpModule
],
providers: [],
bootstrap: [AppComponent]
})
export class AppModule { }`

@nydal91, what's the issue here?

Issue is that is not write like that in the guide in authentication part. Get just error messages if i follow the guide in that part

@nydal91 You'll have to include the exact error messages you're getting for us to be able to help

@nydal91 The Guide has been updated. Not sure, if I got your question correctly, but seems like you're trying to run your code and it's not working.
Why don't you try the below code.
Paste the below code in your  app.module.ts
`
import { BrowserModule } from '@angular/platform-browser';
import { NgModule } from '@angular/core';
import { FormsModule } from '@angular/forms';
import { HttpModule } from '@angular/http';
import { AppComponent } from './app.component';
import {FIREBASE_PROVIDERS, defaultFirebase, AngularFireModule, AuthMethods, AuthProviders, firebaseAuthConfig} from 'angularfire2';
export const firebaseConfig = {
apiKey: "",
authDomain: "",
databaseURL: "",
storageBucket: "",
};
const myFirebaseAuthConfig = {
provider: AuthProviders.Facebook,
method: AuthMethods.Redirect
}
@NgModule({
declarations: [
AppComponent
],
imports: [
BrowserModule,
FormsModule,
HttpModule,
AngularFireModule.initializeApp(firebaseConfig,myFirebaseAuthConfig)
],
providers: [],
bootstrap: [AppComponent]
})
export class AppModule { }
`
and paste the below code in app.component.ts
`
import { Component } from '@angular/core';
import { AngularFire } from 'angularfire2';
@Component({
selector: 'app-root',
template: `
 {{ (af.auth | async)?.uid }}
Login
Logout
  `,
})
export class AppComponent {
  constructor(public af: AngularFire) {}
login() {
this.af.auth.login();
}
logout() {
this.af.auth.logout();
}
}
`
hopefully this should help.


[WFCORE-972] Add handler to process an X-Correlation-ID associated wi…

…th management requests
https://issues.jboss.org/browse/WFCORE-972

Core - Full Integration Build 4349 outcome was FAILURE using a merge of 57bd7cc51913a8d6911b8dde74bdeb7528f9d788
Summary: Compilation error: Compiler (new) Build time: 00:00:20

Full integration - Windows Build 2771 outcome was FAILURE using a merge of 57bd7cc51913a8d6911b8dde74bdeb7528f9d788
Summary: Compilation error: Compiler (new) Build time: 00:01:26

Core - Full Integration Build 5956 outcome was FAILURE using a merge of c026cd48114bcbe3478236f0e1cf407b5020cad0
Summary: Tests failed: 1 (1 new), passed: 4206, ignored: 94 Build time: 01:22:23
Failed tests
org.wildfly.test.integration.elytron.audit.TLSSyslogAuditLogTestCase.testSuccessfulAuth: java.lang.AssertionError: Successful authentication was not logged
	at org.wildfly.test.integration.elytron.audit.AbstractSyslogAuditLogTestCase.testSuccessfulAuth(AbstractSyslogAuditLogTestCase.java:59)

I'm merging this without review because the change, other than the test, is code submitted by Heiko Braun (i.e. I reviewed his change.)


Title on main page is ldjam.com

It should probably say 'Ludum Dare'

Sounds like an easy HTML fix..

The Content (users, pages and post) create there title via their nodes name. ldjam.com is the root nod and so has no name, so it fall back to the web address.
This bit in content probably needs changing, content.js#L25.

Changing it to fallback to "Ludum Dare" seems reasonable as no other nodes should have empty names.

Ludum Dare's true root node is not the root node right now. Symlinks aren't working correctly yet, so I defaulted all sites to the common root (but both Jammer's are disabled so most people didn't likely notice).


⚠ Bofrost.es has degraded performance

In 2bb5599, Bofrost.es (https://www.bofrost.es) experienced degraded performance:

HTTP code: 200
Response time: 1686 ms

Resolved: Bofrost.es performance has improved in 57a03d2 after 6 minutes.


Complex float invalid operation on faster-floats

I don't seem to be able to reduce this one further:
print [as-float32 0.0  lf]
print-wide [as-float32 0.0  as-float32 1.0  lf]

test: function [
	a		[float!]
	b		[float!]
][
	print-wide [a b lf]
	print-wide [as-float32 a  as-float32 b  lf]
]
print-wide [as-float32 0.0  as-float32 1.0  lf]
test 0.0 1.0

0
0 1
0 1
0 1
0
*** Runtime Error 9: float invalid operation
*** at: 08048765h

The crash is caused by return values from prin-float and prin-float32 that are not used, and so accumulate in the x87 stack until exhaustion, causing runtime errors.
Making the compiler smart enough to figure out when the return values are used or not would lead to (probably big) recursive source code static analysis that would be out of the scope of the current Red/System compiler architecture. So the easy fix is to remove return values from prin-float and prin-float32.
Removing also return values from other prin-* function could also be done for sake of consistency.

A possible internal workaround would be to pass the float return values from functions in CPU registers and move them to FPU if they need to be used.

I found a way to avoid the static analysis and discard the return value on FPU stack if not used.

I see the code got smaller by fixing this bug. :-)

Some runtime conversions for returned values were moved at the end of the function body rather than after the call site, hence the code reduction.


Segment Replication - Implement segment replication event cancellation.

Signed-off-by: Marc Handalian<EMAIL_ADDRESS>Description
Segment Replication.  Implement cancellation of replication events.
This PR updates segment replication paths to correctly cancel replication events on the primary and replica.

In the source service (used by primary shards), any ongoing replication event for a primary shard that is sending segments to a replica that shuts down or is promoted as a new primary are cancelled.
In the target service (used by replica shards), any ongoing event for a replica that is promoted as the new primary or is fetching from a primary that shuts down are cancelled.
In the target service, we now handle cancellation initiated by the local node or by handling cancellation initiated by the source node where an ExecutionCancelled exception is sent over the transport layer.
Adds a cancel method to the SegmentReplicationSource interface so that implementations have the opportunity to cancel any ongoing requests and allow for resource cleanup.
Adds a CANCELLED stage to SegmentReplicationState, and transitions the target appropriately.
This change also includes some test cleanup for segment replication to rely on actual components over mocks.

Issues Resolved
closes https://github.com/opensearch-project/OpenSearch/issues/4205
Check List

[x] New functionality includes testing.

[x] All tests pass

[x] New functionality has been documented.

[x] New functionality has javadoc added

[x] Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

  2> REPRODUCE WITH: ./gradlew ':plugins:transport-nio:internalClusterTest' --tests "org.opensearch.transport.nio.NioTransportLoggingIT" -Dtests.seed=1CBD1A01472E341C -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=en -Dtests.timezone=Etc/UTC -Druntime.java=17
  2> com.carrotsearch.randomizedtesting.ThreadLeakError: There are still zombie threads that couldn't be terminated:

* What went wrong:
Execution failed for task ':qa:multi-cluster-search:forbiddenApisTest'.
> Java heap space

REPRODUCE WITH: ./gradlew ':qa:mixed-cluster:v2.3.0#mixedClusterTest' --tests "org.opensearch.backwards.MixedClusterClientYamlTestSuiteIT" -Dtests.method="test {p0=indices.shard_stores/10_basic/basic index test}" -Dtests.seed=115A109D2FC27AAE -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=es -Dtests.timezone=W-SU -Druntime.java=17

Looks like a flaky test again, have run all of these locally with success.

Needed a rebase to apply this fix.

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2037/
CommitID: 1df077d

4: Task failed with an exception.

What went wrong:
Execution failed for task ':qa:full-cluster-restart:forbiddenApisTest'.

Java heap space

FAILURE: Build failed with an exception.

* What went wrong:
A problem occurred configuring root project 'OpenSearch'.
> Could not determine the dependencies of null.
   > Could not resolve all task dependencies for configuration ':classpath'.
      > Could not resolve org.slf4j:slf4j-api:1.7.30.
        Required by:
            project : > com.diffplug.spotless:com.diffplug.spotless.gradle.plugin:6.9.1 > com.diffplug.spotless:spotless-plugin-gradle:6.9.1 > org.eclipse.jgit:org.eclipse.jgit:<IP_ADDRESS>109080827-r
         > Could not resolve org.slf4j:slf4j-api:1.7.30.
            > Could not parse POM https://plugins.gradle.org/m2/org/slf4j/slf4j-api/1.7.30/slf4j-api-1.7.30.pom
               > Could not resolve org.slf4j:slf4j-parent:1.7.30.
                  > Skipped due to earlier error
      > Could not resolve com.googlecode.concurrent-trees:concurrent-trees:2.6.1.
        Required by:
            project : > com.diffplug.spotless:com.diffplug.spotless.gradle.plugin:6.9.1 > com.diffplug.spotless:spotless-plugin-gradle:6.9.1 > com.diffplug.spotless:spotless-lib-extra:2.28.1
         > Skipped due to earlier error
      > Could not resolve org.codehaus.groovy:groovy-xml:3.0.10.
        Required by:
            project : > com.diffplug.spotless:com.diffplug.spotless.gradle.plugin:6.9.1 > com.diffplug.spotless:spotless-plugin-gradle:6.9.1 > com.diffplug.spotless:spotless-lib-extra:2.28.1
         > Skipped due to earlier error

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.

* Get more help at https://help.gradle.org/

BUILD FAILED in 45s
14 actionable tasks: 14 executed
Gradle Check Failed!

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2076/
CommitID: 73a2f66

https://github.com/opensearch-project/OpenSearch/issues/2064

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/2078/
CommitID: 73a2f66

What went wrong:
Execution failed for task ':test:framework:test'.

Process 'Gradle Test Executor 144' finished with non-zero exit value 1
This problem might be caused by incorrect test process configuration.
Please refer to the test execution section in the User Manual at https://docs.gradle.org/7.5.1/userguide/java_testing.html#sec:test_execution

Thanks @mch2 for this PR. This is a critical and important fix.
Please feel free to rebase your branch, it has conflicts with main.

Changelog verifier failiing with - Error: Error: Input required and not supplied: app_id

Updated to not close #4205 with this PR.  In running a new test where the primary is dropped randomly there can still be a leak on the primary side because SegmentReplicationSourceHandler never cancels any ongoing xfers in retryableTransportClient of RemoteSegmentFileChunkWriter.  Will include that fix as a follow up to close out this piece.


1157 remove style elem fix

Handling the DOMNodeRemovedFromDocument for HTMLStyleElements.
This is resolving #1157

@domenic I'm trying your suggestion but I'm getting something wrong. Any hints?

@domenic nevermind, I got it. Silly me :/ But could you check if I'm doing it right?

This looks great to me. Thanks for sticking with us. I'll merge this as soon as I get a free day to do a new jsdom release... soon (TM)...

Cool, thanks!
I would have continued earlier but life happened ;)
Cheers!
Am 12.10.2015 8:00 nachm. schrieb "Domenic Denicola" <
<EMAIL_ADDRESS>
This looks great to me. Thanks for sticking with us. I'll merge this as
soon as I get a free day to do a new jsdom release... soon (TM)...
—
Reply to this email directly or view it on GitHub
https://github.com/tmpvar/jsdom/pull/1251#issuecomment-147476246.

Merged as 5dc416a3aaec1440520ad78945e434c2ceca104b, and expect a 7.0.0 with this fix soon!


feat: add new lights LTP008 and 1742930P7

Adds the CSV data files for the 2 specified models

Hi stijn,
Thanks a lot for the contribution!
Please see my remarks.
Also there is something wrong with 1742930P7/hs.csv.gz file. Every row gives an error in github action.
And we also need to supply a color_temp.csv file for this light, as it is one of the supported color modes. When the user sets the light on color_temperature mode it will give an error now.

As long as the folder names are correct, @bramstroker, this looks good to go

This PR is from a friend of mine. The 1742930P7 has a massive amount of 0.0 readings in lower brightness levels, so we cannot merge this atm.
I have borrowed him my zhurui pr10 which seems to be able to measure these lower watt levels. This is a manual powermeter which only has a display, so I am trying to implement OCR technology in the measure script to be able to read out this display. Making some big progress here, expecting to do some first tests within a few days.

1742930P7 have been removed in favor of #784


Fix license link in README.

Actual link to the license return 404 as it's written licence.

Thanks @cduez


How to Config with vlan ?

hi:
i want use floodlight with xenserver,when i connect to the floodlight ,the xenserver's port with vlan tag ,can not ping 。
can you tell me how to config flow in floodlight?
i add this flow:port 14 has vlan tag:
cookie=0xa00000ec83e777, duration=4114.541s, table=0, n_packets=2786, n_bytes=248759, priority=16383,in_port=14 actions=NORMAL
the port with vlan tag can ping .bug the firewall is failure.
my<EMAIL_ADDRESS>thanks.

@Cookie-Wang The NORMAL action should tell the switch to forward the packet as it would as a learning switch. This sounds like it's working if you can ping and that's the type of flows you are inserting.
What about the firewall isn't working? What exactly are you trying to accomplish? (Sorry for getting to this so late...Feel free to use the mailing list in the future for a more rapid reply. More people are active on that than on the GitHub issues.)

@Cookie-Wang have you resolved the issue?

@Cookie-Wang I'm going to close this issue to keep the list clean. Please reopen if you have another question.


docs: Fix a link in contribution guide

Fixes #2492

but not sure I want to spend more time there!

I have time management issues 😅 just trying to fix this led me to update Docsy and Hugo versions because I typed hugo server with my local latest version and the build failed, so I updated everything in https://github.com/cilium/tetragon/pull/2497.


Fix Partition All Docs Query With Keys

Overview
This PR addresses a bug that is described in issue #2496 where posting an all docs query to a partition endpoint with keys that are not a part of the partition also gets returned.
Testing recommendations
Consider the following documents, each in their own partition on a database named test.
"rows": [
        {
            "id": "part1:doc1",
            "key": "part1:doc1",
            "value": {
                "rev": "5-aa9d2b82de6db1e61fba8e4057299d21"
            }
        },
        {
            "id": "part2:doc1",
            "key": "part2:doc1",
            "value": {
                "rev": "3-a707bb62c86bd430389e4d596aeb6084"
            }
        }
]

When we run the following query against the part2 partition:
curl -X POST http://<IP_ADDRESS>:15984/test/_partition/part2/_all_docs \
    -H 'Content-Type: application/json' \
    -d '{"keys":["part1:doc1", "part2:doc1"]}'

We get both documents from both partitions, but what we expect should look like this
"rows": [
        {
            "key": "part1:doc1",
            "error": "not_found"
        },
        {
        "id": "part2:doc1",
            "key": "part2:doc1",
            "value": {
                "rev": "3-a707bb62c86bd430389e4d596aeb6084"
            }
       }
    ]

The code in this PR fixes this issue and returns the results we expect.
Related Issues or Pull Requests
Issue #2496
Checklist

[x] Code is written and works correctly
[x] Changes are covered by tests
[n/a] Any new configurable parameters are documented in rel/overlay/etc/default.ini
[n/a] A PR for documentation changes has been made in https://github.com/apache/couchdb-documentation

Thanks for this, this looks decent overall, especially for a first contribution :)
I left a few minor notes on the code. I just have one question about the approach taken here.
The way I read it: passing keys to match into the query will return result rows we do not want. So this PR starts preventing those result rows from being returned.
Would another option to solve this be making it so that we don’t read those rows in the first place? That would make this a performance improvement too.
We might even be able to do this at the request validation stage: if keys includes a partition that is not the partition of the URL being queried, we return a 400 error with a message that all keys must match the partition provided in the URL.
Just spitballing here tho. Nice work!


utility-types is missing in peer dependencies

Hello :)
In the current 2.0.0-beta.1 you are using utility-types, which is only used as dev-dependency. Please consider to move it in the peer-dependency or dependency config in the package json.
Following error occurs otherwise:
TypeScript error in /app/frontend/node_modules/react-router-last-location/dist/withLastLocation.d.ts(7,186):
Cannot find module 'utility-types'.  TS2307
5 |     lastLocation: ReturnType<typeof getLastLocation>;
6 | }
7 | declare const withLastLocation: <WrappedProps extends WithLastLocationProps>(WrappedComponent: React.ComponentType<WrappedProps>) => React.ComponentClass<Pick<Pick<WrappedProps, import("utility-types").SetDifference<keyof WrappedProps, "history" | "location" | "match" | "staticContext" | "lastLocation">> & RouteComponentProps<{}, import("react-router").StaticContext, any>, Exclude<import("utility-types").SetDifference<keyof WrappedProps, "history" | "location" | "match" | "staticContext" | "lastLocation">, "history" | "location" | "match" | "staticContext">>, any>;

Hey @valentinpalkovic ,
Many thanks for reporting the issue.
I published 2.0.0-beta.2 which should solve the issue. You don't have to install utility-types, just upgrade react-router-last-location.

I'm closing issue because this package doesn't depend anymore on utility-types.


python310Packages.spacy: 3.4.4 -> 3.5.0

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python310Packages.spacy is: Industrial-strength Natural Language Processing (NLP)
meta.homepage for python310Packages.spacy is: https://github.com/explosion/spaCy
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact

Checks done (click to expand)

built on NixOS
Warning: a test defined in passthru.tests did not pass
found 3.5.0 with grep in /nix/store/c7kc19yv5626y14yzyjkqkfi59454srg-python3.10-spacy-3.5.0
found 3.5.0 in filename of file in /nix/store/c7kc19yv5626y14yzyjkqkfi59454srg-python3.10-spacy-3.5.0

Rebuild report (if merged into master) (click to expand)

3 total rebuild path(s)

3 package rebuild(s)

First fifty rebuilds by attrpath
python310Packages.spacy
python310Packages.spacy-transformers
python310Packages.textacy

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/c7kc19yv5626y14yzyjkqkfi59454srg-python3.10-spacy-3.5.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python310Packages.spacy https://github.com/r-ryantm/nixpkgs/archive/372a4b9e5c71ff5c21ef974cbc8a13b00c863527.tar.gz

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/c7kc19yv5626y14yzyjkqkfi59454srg-python3.10-spacy-3.5.0
ls -la /nix/store/c7kc19yv5626y14yzyjkqkfi59454srg-python3.10-spacy-3.5.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

Maintainer pings

Closed by #214332.


use only git fetch

It looks like a hack to just get the PR branch, rather than the original branch and then PR branch is possible.
Right now,
git clone https://github.com/some/repo
git fetch pull/ID/head:pr-ID
git co pr-ID

It looks like we could do,
git init
git fetch https://github.com/some/repo pull/ID/head:pr-ID
git co pr-ID

It would limit a needless checkout. My only wonder is anyone using it?

Probably also worth fetching the base branch too, for those who check for differences between the PR and base (actual diffs, log entries, etc).

@jdub, I know that is something that you can do, but wasn't necessarily documented feature. :wink: Now creating an interface to support this behaviour is something I wouldn't mind adding.
I created this issue because I wanted to propose a more optimized way of pulling the PR, therefore causing less clones.

@jdub, I'm thinking of providing an option to pull down the base branch, too. I've got some pipelines where doing two git clones on a repo takes forever, mainly because of submodules.

I'm actually going to close this. It appears that depth flag wasn't in all the right places, thanks to some PRs this is fixed.

Great changes!


update mkdirp and yargs-parser dep to fix vulnerabilities (6.2 release)

Description of the Change
Mocha 6.2.x is latest version supporting Node versions below 12, therefor used by lots of projects needing this.
This version depends on two different packages which has security vulnerabilities: mkdirp and yargs-parser. This PR updates these two dependencies to the next higher version with fixes for these vulnerabilities and nothing else.
Alternate Designs
Decided for this change to only change the minimum amount required for vulnerabilities flagged in production dependencies.
Why should this be in core?
Without this PR all builds of projects using mocha and requiring successful running vulnerability checks like `npm audit´ are failing.
Benefits
npm audit not failing for projects using mocha.
Possible Drawbacks
none
Applicable issues

Is this a breaking change (major release)?  NO
Is it an enhancement (minor release)?  YES
Is it a bug fix, or does it not impact production code (patch release)? YES

New minor version should be released after accepting this PR.
References

minimist (dependency of mkdirp): https://www.npmjs.com/advisories/1179
yargs-parser: https://snyk.io/vuln/SNYK-JS-YARGSPARSER-560381

Thanks,
Stefan Seide

Coverage increased (+0.07%) to 92.588% when pulling 3d9b6644d5d8fefe974a703fe649ca654da45a59 on sseide:fix_mkdirp into 957c7ebd8dc4e1cffda8a22509357660f63590a4 on mochajs:release/v6.2.x.

This statement is not correct:

Mocha 6.2.x is latest version supporting Node versions below 12

Which Node versions are you testing with?

looking at the release notes and changelog. There  are ony tags for 6.2 releases and 7.1 And 7.0 got no security update lately, only the 7.1 line.
7.1.0 states prominently in the changelog "Node.js only v12.11.0 and above". But (on a third read) it may want to say 12.x for some special conditions and not globally... So please correct me if im wrong...

looking at the release notes and changelog. There are ony tags for 6.2 releases and 7.1 [...]

There are dozens of tags listed, I don't know where you are looking at.

"Node.js only v12.11.0 and above" [...]

This is valid for ESM support.

looking at the release notes and changelog. There are ony tags for 6.2 releases and 7.1 [...]

There are dozens of tags listed, I don't know where you are looking at.

Here i was referring to the git repository having a "release/6.2.x" and "release/7.1.x" but no "release/7.0.x" branch. Therefor this made me think the 7.0 line is gone an no maintenance or something  else any further. And 7.1 has the statement about Node 12 below ..

"Node.js only v12.11.0 and above" [...]

This is valid for ESM support.

Ok, as i understand now this min requirement is not global for mocha itself.
Shall i close the PR than? 6.x line of releases is out of maintenance? Fine for me when upgrade to latest 7.1 is possible with Node 10.x

I would like to also voice my support here - would be nice to backport security vulnerability fixes to the 6.x versions, for those projects using v6.

@sseide No, please leave the PR open for now.
@karlhorky we don't maintain LTS releases. The last release (v7.1.1) is normally the only maintained version. We haven't decided yet, wether we backport the security fix to v6.2.2.

Calling npm audit --production or running the snyk.io vulnerability checks for production dependencies only does not show vulnerabilities anymore. Everything is green.
$ npm audit --production

                       === npm audit security report ===

found 0 vulnerabilities
 in 169 scanned packages

Checking with devDependencies (e.g. npm audit) its reporting much lower problems now. But having devDependncies without errors is not really needed. The production dependencies are much more important...
$ npm audit

                       === npm audit security report ===
...
found 47 vulnerabilities (12 low, 2 moderate, 33 high) in 29906 scanned packages
  run `npm audit fix` to fix 15 of them.
  1 vulnerability requires semver-major dependency updates.
  31 vulnerabilities require manual review. See the full report for details.

@sseide your last commits don't make any sense.
Please read npm.
Key statement: One key detail about package-lock.json is that it cannot be published [...]

@juergba yes you are right. Was too late this day and forgot this is not published to npm.
Just seen this file is part of the git-repo (see https://github.com/mochajs/mocha/tree/release/v6.2.x) and people checking it out will get dozen of old dependencies..

but at least the changes to the package.json will fix problems for others, as these versions listed as "dependencies" are free of known security problems. Therefore you do not have to backport your code changes to replace mkdirp package with other implementations... It may be less intrusive.

We published<EMAIL_ADDRESS>


not working after FilePicker.Default.PickAsync

hi, i have an issue
ScreenSecurity.Default.ActivateScreenSecurityProtection();
var result = await FilePicker.Default.PickAsync(options);

this code works for me.
but
var result = await FilePicker.Default.PickAsync(options);
ScreenSecurity.Default.ActivateScreenSecurityProtection();

screen security will not activate after await FilePicker.Default.PickAsync.
can u help me?
everything is ok on windows. my problem is on android
i want to activate screen security on some file types not for all of them.

Hi @vahidsoufi, can you explain further what are you trying to archive? Also, check if the awaited call to FilePicker is not breaking at some point which can mess with the execution of the code after that line.

thx for reply bro,
everything is ok on windows.
FilePicker is awaited, file picks truely, after that if file extension is mp4 i want to activate security plugin, but not working on android. at this moment i didnt test it on ios.
therefore i did my job with window flags secure mode. but thats wonder for me why pluging not working after FilePick. i used .net7
can u test it?
another question?
can you do anything for mac? do u have any solution?
at the end, thanks for your  nice efforts

Hi @vahidsoufi, hope you're doing well.
I've tested this and you are correct, for some reason, it doesn't work in that order, I didn't investigate this issue further, but if you use the plugin as you mention here, it works correctly:

ScreenSecurity.Default.ActivateScreenSecurityProtection();
var result = await FilePicker.Default.PickAsync(options);
this code works for me.


usage area

Hello
I have a question of use does ln link work on Debian 10 or 11 machine? (example proxmox or proxmox virtual machine)
If yes I have a failure on the command pip3 install -U lnxlink and how to fix this?

You should be able to install on any linux distro, including Debian.
I haven't seen this error before, but you could try this:
sudo pip3 install -U pip
pip3 install -U lnxlink

If it doesn't work send me the result of this command
pip3 -V

You could also try installing another python package to check if it works:
pip3 install PyYAML

Good evening
So I just placed the two commands and I still have an error on the second command

For pip3 -v
pip 23.0.1 from /usr/local/lib/python3.9/dist-packages/pip (python 3.9)

I haven't tried it on raspberry, but you could try this:
pip3 uninstall gi pgi
sudo pip3 uninstall gi pgi
sudo apt install python3-gi

When I get the chance, I will try it myself on a raspberry pi 4.

I've tried installing it a fresh installation of raspbian lite on a raspberry pi 4 and it works for me.
The only changes i've made are to disable these from the config.yaml file:

camera_used
idle
nvidia_gpu


1.1.1

What does this PR do?
Prepare the release of 1.1.1 version.

what about this commit? https://github.com/DataDog/dd-native-iast-taint-tracking-js/commit/d1080658529ef369bb5b90e677d86a759bc223bf

The commit d108065 contains a change for codeql analysis action, a job that is targeted to master and feature/iast-chapter-1 branches, so I didn't think it should be included in a v1.x release.


clean imports in tests

What does this PR do?
clean the unused imports in tests
PR review
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃

same here? https://github.com/PyTorchLightning/pytorch-lightning/pull/3777#issuecomment-702685136


unable to connect snowflake

Bug Report --   after moving drivers to CE edition, there is no method to specify SAML authentication.
https://docs.snowflake.net/manuals/user-guide/jdbc-configure.html
(key -- authenticator)

Thanks for bug report.


[Bug]: After clicking on like button /unlike button the count is static

Is there an existing issue for this?

[X] I have searched the existing issues

Describe the bug
Making functionality for like button
Expected behavior
the number incerements or decrements based on likes and dislikes
Add ScreenShots

On which device are you experiencing this bug?
Windows
Record

[X] I have read the Contributing Guidelines
[X] I'm a GSSOC'24 contributor
[X] I have starred the repository

/assign it to me

It doesn't work on previous posts, when this feature is introduce after that the post which is created it will work fine and currenlty it is working perfectly.

excellent ...


Queue size

Hi
I have a question regarding the queue size. After some time the queue stopped and when I ran the program again, it went through the messages it went with the first queue and did this time nothing. Is there a way to increase the lifespan of a queue by editing the code? I don't have any experience with programming, so I'm quite behind when it comes to code reading.
Thank you for your time.

Hi
I have a question regarding the queue size. After some time the queue stopped and when I ran the program again, it went through the messages it went with the first queue and did this time nothing. Is there a way to increase the lifespan of a queue by editing the code? I don't have any experience with programming, so I'm quite behind when it comes to code reading.
Thank you for your time.

Same question. It scans and deletes a certain number of texts, and then scans the first batch again and scrolls up slowly to delete the new ones.

I am not sure what your problem is… You could maybe open another issue using the issued template :


Add solution values for linear expressions

Variables have computed values in the solution after a successful model solve, but linear expressions do not.
It would be great for solution values to be computed for any linear expressions that are stored in memory at solve time. Or, like Pyomo, to have linear expressions added to the model (add_linexpr?) and then have values computed for those expressions on a successful solve.

Hey @brynpickering, thanks for your comment. This would actually be quite nice. We could add a solution attribute to the LinearExpression/QuadraticExpression class. Since these are always referenced to a model instance, it will be easy to retrieve/process the values.

I was about to open an issue before I saw this already existing one. The proposed feature would be a great benefit for us, since our models consist of a composition of many linear expressions, but each expression itself is an important part of subsequent analyses.

I totally agree, the feature is coming in #249 :)


Extend WebGL color operator

This improvement is inspired by issue https://github.com/openlayers/openlayers/issues/14350
It solves the overhead calculations for grayscale colors in WebGL style expression where same value was calculated three times and it also caused troubles on legacy devices.
I updated the code, comments for the docs and examples.

I do not see anything to improve now, this can be merged.


Replicas > 1 is allowed with sqlite driver and it shouldn't be.

We need to check for replicas > 1 when the sqlite driver is being used and block install/upgrades. It is not a supported or even workable configuration.

Desired behavior here is to issue an error statement that "The SQLite3 Datastore cannot support more than one Server instance, please select another datastore or set the number of server instances to one"
That way the error message embeds the fix.


nrf52_pca10040:tests/bluetooth/init/test_controller_dbg fails build with CONFIG_USERSPACE=y

Very strange error on in nrf52_pca10040:tests/bluetooth/init/test_controller_dbg:
/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/subsys/bluetooth/host/smp.c: In function ‘sign_test’:
/home/buildslave/src/github.com/zephyrproject-rtos/zephyr/subsys/bluetooth/host/smp.c:3864:1: error: r7 cannot be used in asm here
 }
 ^
zephyr/subsys/bluetooth/host/CMakeFiles/subsys__bluetooth__host.dir/build.make:254: recipe for target 'zephyr/subsys/bluetooth/host/CMakeFiles/subsys__bluetooth__host.dir/smp.c.obj' failed
make[3]: *** [zephyr/subsys/bluetooth/host/CMakeFiles/subsys__bluetooth__host.dir/smp.c.obj] Error 1

I believe I know what this is.
Most ARM use the latest AAPCS which does not keep a frame pointer. There is impossibly convoluted logic and the exidx section used for stack unwinding.
I suspect this board uses the older version, where r7 is used to store the frame pointer, and GCC does not like this. Might have to do this directly in assembly.

@andrewboie where is the aapcs version selected in the board? I don't think this does anything differently than other boards, the difference is perhaps that we are the only ones to build the Bluetooth host and controller together.
CC @pizi-nordic

The AAPCS designated r11 as a frame pointer, so we are still in the fog. I will try to reproduce this issue and try to find the cause.

I traced it down.

The AAPCS defines r11 as a frame pointer for long mode, however in thumb a r7 is used (as lower registers might be used with more instructions).

The 09a8810b339b8266f835ea29f863a9cb825e486a adds the r7 to the clobber list of  _arch_syscall_invoke*() calls. The r7 must be listed there as it is overwritten by the _do_syscall.

The GCC prints the mentioned error if r7 is used by assembly code in the function that has frame pointer.

So, as long as there is no frame pointer (-O1 or higher or -fomit-frame-pointer passed and no local data allocated on stack) in the function, everything will work OK. However a more complex code (using frame pointer, like the mentioned sign_test) will be not able to invoke any syscall.
IMHO this the code introduced in 09a8810b339b8266f835ea29f863a9cb825e486a should be updated in a way which ensure r7 preservation during syscall.

Yeah I'll take shortly.  We moved from using stack to using regs for these due to unaligned stack issues.  I didn't know the thumb mode did things a little different.

So the problem with fixing this I think goes back to the way args are passed.  So originally we used some stack space to store parameters to avoid using the r4/r5/r6/r7.  This didn't work out well due to the compiler sometimes creating unaligned stacks which threw off our stack retrieval of parameter information.
So to get around that issue, we move everything to use r0/r1/r2/r3/r4/r5/r6/r7 to pass information.  Clearly that isn't going to work in some cases.
So what if we instead pass in stack ptr to the syscall that points directly to the syscall frame that describes the parameters.  This is like the original solution but works around the stack padding that may or may not occur due to the alignment of the stack.  So we'd only use r0/r1/r2/r3 at most to pass parameters.  Any additional ones would rely on stack storage and the passing of the stack ptr in r0, r1, r2, or r3.

The problem is that this calling convention uses r7. So for example the r0/r1/r2/r3/r4/r5/r6/r12 should work and you will not need to create stack area for arguments. What do you think?

R7 is for the saved link register.  We need that to be able to return back to the original caller.

Summarising:

We have to remove r7 from clobber list.
To do so, we have to preserve r7 during sycall. At the moment the r7 holds saved LR (set in _do_syscall, used in _arm_do_syscall).

I have created a PR https://github.com/zephyrproject-rtos/zephyr/pull/8103, which fixes this problem by using r8 instead of r7 to hold saved LR.


Suncharn

Consolidate 2 Boxes classes into 1 Box class
Created teeth_arrangement script & calls the script for nms
Change the input file to be 200 items instead of sample file.
Use TensorFlow nms library instead of tom hoag's nms library.
Note: to use TensorFlow, we need python 64bits version 3.5-3.8.

Merging this for NMS, couldn't run Visualizer but it's okay since we don't have the large imgset.


Make camelCasing optional

Would be nice if we could make camelCasing optional:
schema = strawberry.Schema(query=Query, auto_camelcase=False)
https://github.com/strawberry-graphql/strawberry/issues/36

@patrick91 is there an update on this :O

@patrick91 is there an update on this :O

sorry I've been quite busy in the last months, I'll make sure this happens over next week 😊

Any news?

@zerocewl I'm aiming to finish it this week :) I made some changes to make this easier in #1044 so hopefully I won't find any blockers :)

Sorry this took more than expected, we want to release a big refactor before doing it! Now it is done and should be released soon!
Thanks for waiting!


Dl/fastkronmatrix

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 79.48%. Comparing base (0f73d9d) to head (709829b).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master       #8      +/-   ##
==========================================
+ Coverage   78.82%   79.48%   +0.66%
==========================================
  Files           3        3
  Lines         373      385      +12
==========================================
+ Hits          294      306      +12
  Misses         79       79

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


⚠ Processors has degraded performance

In 52ce9d8, Processors ($STATUS_URL) experienced degraded performance:

HTTP code: 200
Response time: 76 ms

Resolved: Processors performance has improved in 33dd7f2.


updated version of installation instructions

Hey dmitriiabramov,
the blind copy/paste into my package.json didn't work, when I try to use it. So I figured out that the version in the installation guide was not up to date.
What do you think?
Greets
FQ400

absolutely!
thank you for taking care of that :)


Java solution to the Luhny Bin

Here's my Java solution to the Luhny Bin problem. It memoizes the "Luhny Sums" (odd and even) of potential credit card numbers at each character offset in the input stream. It uses the lookup table to compute the check values for 14 through 16 digit numbers ending at any given character and masks accordingly.
Some potential improvements:

Store information in the mask step to eliminate the need to remask overlapping regions.
Save memory by not storing the entire lookup table in memory. Each time a non-credit-card character is encountered, the table can be cleared. This is only necessary if long inputs are expected.

All in all, a fun problem. Thanks!

Pull request now includes some additional tests:

Don't assume CC #'s will be in 4 digit segments (if delimited).
Don't assume that delimiters will only be one character long.


Add a message field to the issueRecorded event in the experimental JSON event stream

Description
Experimental event streaming support was added in #286. Issues are reported as individual events with richly structured JSON that describe the failure. This is great for tools that need to work with and transform raw issues, but many tools will just want to use the human readable failure string, i.e: Expectation failed: 1 == 2.
Swift-testing has done a lot of work to build up a nice message, and it would be nice to avoid reimplementing this in other tools (and keeping it up to date).
Feature: Add a new message field to the issueRecorded event that provides the human readable failure string. Bonus points if there is a markdown version of the string, as many tools can render markdown directly.
Expected behavior
No response
Actual behavior
No response
Steps to reproduce
No response
swift-testing version/commit hash
No response
Swift & OS version (output of swift --version && uname -a)
No response

As of right now, the experimental event stream just encodes the fields of the events themselves. In order to make a stable API/ABI, we likely need to change that to a formal JSON schema with dedicated and intentional content, not just "whatever JSONEncoder produced."


add router to middleware arguments

fixes #14

@aaronshaf This was pretty simple. I would have loved to have someone contribute this, but honestly this project doesn't have a big enough audience to wait for that.

:+1:


Is there have a setting item to set zh-CN or en-US?

I can't find a setting item to set it, it looks like following system language

There is no setting at present.
It uses the language of display language of VSCode.


nature of code book "short links"

The nature of code book (https://github.com/nature-of-code/noc-book-2023/) will contain many "short urls" to coding train videos in the book. A running list of them is here, filing this issue to remember to add all of these as redirects on thecodingtrain.com.

These have been completed!


Long term names => loss of query details

From @dosumis on October 24, 2017 17:45
For example, ("clustered") is clipped from the end of Query for:
neurons with some part in the antennal mechanosensory and motor center (clustered)
Suggested Fix: Pop up the full text when mousing over the query.
Copied from original issue: VirtualFlyBrain/geppetto-vfb#83

From @dosumis on October 24, 2017 18:48

From @dosumis on November 21, 2017 14:35
Poss solution - float over => full text.

From @dosumis on November 21, 2017 14:36
Or clip the name with ... ?


TCRS derive now runs in parallel

As far as I can see runtime changed from 2-3 hours to 5 minutes per seed

Both I and @chunkinaround have tested this and it speeds up this run significantly!


Gallery Images are not being optimized and have lower quality than originals

Describe the bug
I'm using the gallery shortcode on multiple pages in my website, and although the full images being used have good quality in their original size, when placed in a gallery they seem to be blurry and weirdly scaled.
I have disableImageOptimization = false set in the config but looking at the urls for gallery images I can see they are not going through image optimization, contrary to what happens to all other images placed via the figure shortcode for example.
To Reproduce
Steps to reproduce the behavior:

Go to 'https://joaocastro.me/projects/digital-avatars-mobile-app/' (for example);
Sroll down and click on any image on a gallery to zoom;
Right-click and open the original image in a new window to see the difference in quality;

Expected behavior
Ideally all images should all run through image optimization process, or even if this would to be disabled, the original image scalling in the gallery should not produce blurried and lower quality assets.
Screenshots
Blurry image in a gallery:

Original image opened in a separate tab:

Tested locally with:
hugo v0.125.7+extended
Blowfish v2.66.0
Am I missing something on my installation or should the gallery shortcode be used in other form?
Thanks in advance for any help.

Which browser?
Please share the repo

Thank you for your reply @nunocoracao !
This issue was currently detected on Safari 17.4 (196<IP_ADDRESS>.12). Have tested also on Firefox 125.0.3 and everything seems to be working ok (haven't been able to test on other browsers yet).
I'll be duplicating a repo to share with you here shortly, currently I'm hosting it on GitLab.
The issue can also be detected with Safari on your current demo site:
https://blowfish.page/docs/shortcodes/#gallery
Thank you again for your reply.

Hello @jcastro-design sorry I won't have tie to look at this one any time soon. I've experienced the same problem and it seems to happen only in Safari. I do not have time to develop/maintain a browser-specific hack here.

I understand @nunocoracao , thanks for the update. If somehow I'm able to stumble upon a possible fix I'll share it here.

Will close the ticket since this seems to be on Safari side not the theme.


Clarify empty string is the same as false when skipping

Clarifying the use of "" within the skip command section. Wasn't sure whether to add this as a sentence or as an example case, but I decided it wasn't clear what the example case results in.

Great addition! Thanks for taking the time to PR @benclive 😄🙌🏻


Upgrade from HDD to SSD in k8s deployment on Azure

Blocked on this because ACS does not currently support Premium_SKU in k8s Storage Class.

Azure supports this. Currently we are using only Standard_D2_v2 and the storage account with Standard LRS in our deployment model. However, if we want we can create a Premium LRS storage account on Azure and use that for our deployment.
Some references:
Kubernetes Github
General purpose Linux VM sizes
High-performance Premium Storage and managed disks for VMs
Recent blog
Do we need to introduce this in our deployment guide now?

@muawiakh I think we can close this issue now. The (optional) use of Premium storage on Azure is documented at:
https://docs.bigchaindb.com/projects/server/en/latest/production-deployment-template/node-on-kubernetes.html?highlight=Premium#step-10-create-kubernetes-storage-classes-for-mongodb


2.1.4. Implement R-LWE plaintext polynomial encodings for medium and high resolution iris codes

This is deliverable 4 in milestone 1 of phase 2.
Polynomial encodings transforming low and high resolution iris codes to R-LWE plaintext polynomials.

[x] https://github.com/Inversed-Tech/eyelid/issues/81
[ ] https://github.com/Inversed-Tech/eyelid/issues/88
[ ] https://github.com/Inversed-Tech/eyelid/issues/89

The only task left here is #89, so I’m going to close this ticket.


[TF 2.0] tf 2 around 40 times slower than tf 1 for unrolled taylor series

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
TensorFlow installed from (source or binary): pip
TensorFlow version (use command below): 1.13.1
Python version: Python 3.6.7
CUDA/cuDNN version: 10.0/7.4
GPU model and memory: GeForce GTX 1060, Compute Capability 6.1, 6GB RAM

This issue follows on from https://github.com/tensorflow/tensorflow/issues/26807
Describe the current behavior
The code linked at the bottom calculates a matrix exponential (expm) using a taylor series. It outputs the following when running with the latest TF1 and TF2 installations from pip:
(tf1) $ python expm.py
BENCHMARKS:
tf took 0.1590 seconds for 25 iterations
taylor took 0.0158 seconds for 25 iterations

(tf2) $ python expm.py
BENCHMARKS:
tf took 0.7530 seconds for 25 iterations
taylor took 1.0743 seconds for 25 iterations
taylor_v2 took 0.6025 seconds for 25 iterations
PROFILES:
profile for taylor_v2
         2251 function calls in 0.634 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
       25    0.000    0.000    0.634    0.025 expm.py:74(<lambda>)
       25    0.000    0.000    0.634    0.025 /home/jeff/.virtualenvs/tf2/lib/python3.6/site-packages/tensorflow/python/eager/def_function.py:400(__call__)
       25    0.000    0.000    0.634    0.025 /home/jeff/.virtualenvs/tf2/lib/python3.6/site-packages/tensorflow/python/eager/function.py:1267(__call__)
       25    0.000    0.000    0.632    0.025 /home/jeff/.virtualenvs/tf2/lib/python3.6/site-packages/tensorflow/python/eager/function.py:542(_filtered_call)
       25    0.000    0.000    0.632    0.025 /home/jeff/.virtualenvs/tf2/lib/python3.6/site-packages/tensorflow/python/eager/function.py:560(_call_flat)
       25    0.000    0.000    0.630    0.025 /home/jeff/.virtualenvs/tf2/lib/python3.6/site-packages/tensorflow/python/eager/function.py:379(call)
       25    0.000    0.000    0.630    0.025 /home/jeff/.virtualenvs/tf2/lib/python3.6/site-packages/tensorflow/python/eager/execute.py:33(quick_execute)
       25    0.630    0.025    0.630    0.025 {built-in method _pywrap_tensorflow_internal.TFE_Py_Execute}
       25    0.000    0.000    0.001    0.000 /home/jeff/.virtualenvs/tf2/lib/python3.6/site-packages/tensorflow/python/eager/function.py:1505(_maybe_define_function)
   ...

I've included some profiling information for the function of interest (taylor_v2 in TF 2) which shows that it spends all the time in _pywrap_tensorflow_internal.TFE_Py_Execute.
Describe the expected behavior
I expect TF 1 taylor (which uses tf.Session.run) and TF 2 taylor_v2 (which uses tf.function) to have similar performance.
Code to reproduce the issue
import tensorflow as tf
import numpy as np
import time as tm
import cProfile as pr
import io
import pstats as ps

MATRIX_DIM = 1000
TAYLOR_SUM = 30
EPS = 1e-2
WARMUP = 10
ITERATIONS = 25

def benchmark(name, fn):
    for _ in range(WARMUP):
        value = fn()
    start = tm.time()
    for _ in range(ITERATIONS):
        value = fn()
    end = tm.time()
    runtime = end - start
    print(f"{name} took {runtime:.4f} seconds for {ITERATIONS} iterations")
    return value

def profile(name, fn):
    for _ in range(WARMUP):
        value = fn()

    profile = pr.Profile()
    profile.enable()

    for _ in range(ITERATIONS):
        value = fn()

    profile.disable()
    s = io.StringIO()
    sortby = "cumulative"
    stats = ps.Stats(profile, stream=s).sort_stats(sortby)
    stats.print_stats()
    print(f"profile for {name}")
    print(s.getvalue())

    return value

def taylor_expm(x, n):
    x_0 = tf.eye(tf.shape(x)[0], dtype=x.dtype)
    x_i = x
    y = x_0 + x_i
    for i in range(2, n + 1):
        x_i = (x_i @ x) / tf.cast(i, x.dtype)
        y = y + x_i
    return y

np.random.seed(42)

x = tf.constant(np.random.uniform(-0.5, 0.5, [MATRIX_DIM, MATRIX_DIM]), tf.float32)

if tf.__version__.startswith("2"):

    @tf.function
    def taylor_expm_v2(x, n):
        return taylor_expm(x, n)

    print("\nBENCHMARKS:")
    tf_expm = benchmark("tf", lambda: tf.linalg.expm(x))
    my_expm = benchmark("taylor", lambda: taylor_expm(x, TAYLOR_SUM))
    my_expm_v2_b = benchmark("taylor_v2", lambda: taylor_expm_v2(x, TAYLOR_SUM))

    print("\nPROFILES:")
    my_expm_v2_p = profile("taylor_v2", lambda: taylor_expm_v2(x, TAYLOR_SUM))

    np.testing.assert_allclose(tf_expm.numpy(), my_expm.numpy(), atol=EPS)
    np.testing.assert_allclose(tf_expm.numpy(), my_expm_v2_b.numpy(), atol=EPS)
    np.testing.assert_allclose(tf_expm.numpy(), my_expm_v2_p.numpy(), atol=EPS)
else:
    tf_expm_ = tf.linalg.expm(x)
    my_expm_ = taylor_expm(x, TAYLOR_SUM)

    with tf.Session() as sess:
        print("\nBENCHMARKS:")
        tf_expm = benchmark("tf", lambda: sess.run(tf_expm_))
        my_expm = benchmark("taylor", lambda: sess.run(my_expm_))

    np.testing.assert_allclose(tf_expm, my_expm, atol=EPS)

Ah!
This is the subtlest and most common trap when it comes to profiling tensorflow: if you use a tf.constant to represent a graph's inputs, by default TF will constant fold the entire graph out of existence.
So if you replace x = tf.constant with x = tf.placeholder and use a feed_dict you'll get something closer to the real performance numbers, and session.run is no faster than tf.function.

@alextp that's amazing - thank you!


opensnoop.bt: no open syscall on aarch64

What reproduces the bug?
On aarch64, opensnoop fails to run:
# uname -m
aarch64
# opensnoop.bt -b
/tmp/opensnoop.bt:22-24: ERROR: tracepoint not found: syscalls:sys_enter_ope

That is because aarch64 has no open syscall, as a new arch they only implement the openat* variants, so there will obviously be no associated tracepoint. See iovisor/bcc#3344 for discussions around the libbpf-tools variant of opensnoop, where probe loading will likely be disabled based on arch.
Removing sys_enter/exit_open tracepoints from opensnoop.bt obviously works but I'm curious if bpftrace has a way to handle that kind of things gracefully, or if we'd have to fall back to things like cmake generating an appropriate opensnoop.bt based on target arch? which isn't exactly pretty...
(Bonus question, I didn't take the time to try yet but if someone knows -- arm32 has an open syscall and aarch64 has 32bit emulation, so it should be possible to run a 32bit program that actually uses open there. How will it show up in tracepoints? Looking at the code it should go through COMPAT_SYSCALL_DEFINE3(open... which just calls do_sys_open which doesn't seem to go through any tracepoint unlike the SYSCALL_DEFINEx, so there will just not be caught?)
bpftrace --info
System
  OS: Linux 5.4.47-1-at #2-Alpine SMP PREEMPT Fri May 7 00:27:04 UTC 2021
  Arch: aarch64

Build
  version: v0.11.3
  LLVM: 11
  foreach_sym: yes
  unsafe uprobe: no
  bfd: no
  bpf_attach_kfunc: yes
  bcc_usdt_addsem: yes
  bcc bpf_attach_uprobe refcount: yes
  libbpf: yes
  libbpf btf dump: yes
  libbpf btf dump type decl: yes

Kernel helpers
  probe_read: yes
  probe_read_str: yes
  probe_read_user: no
  probe_read_user_str: no
  probe_read_kernel: no
  probe_read_kernel_str: no
  get_current_cgroup_id: yes
  send_signal: yes
  override_return: no

Kernel features
  Instruction limit: 1000000
  Loop support: yes
  btf: yes

Map types
  hash: yes
  percpu hash: yes
  array: yes
  percpu array: yes
  stack_trace: yes
  perf_event_array: yes

Probe types
  kprobe: yes
  tracepoint: yes
  perf_event: yes
  kfunc: no

We don't really have a solution for it. As script go through the C pre processor you might be able to do some #ifdef magic although thats a bit hacky.
I guess having an easy way to do arch dependent things could be useful, although I don't see a lot of usecases yet.
Or maybe we can expand the probe name matching. If you could write something like tracepoint:syscalls:sys_enter_open(at)? {} . The current wildcard matching only runs on probes that it finds, so thats what you want. The only issue is that sys_enter_open* matches too much, so basic regex support might be good

pattern matching is elegant, I like it! But there's also the problem of not having any match, or if e.g. sys_enter_open's probe and sys_enter_openat probe contents would become different you'd need to split it in two sections and in that case the sys_enter_open probe would have code but no associated entry point, which currently errors out.
What I had suggested early in the libbpf case was to just make a probe optional, for bpftrace that would be some syntax like tracepoint:syscalls:sys_enter_open? { ... } that hint that if the probe isn't found it's ok to just continue - since we list available probes for pattern matching we can check if the name exists and act on it so that should be possible. (would need a syntax different from ? if we ever have basic regex support)

Closing for now. Feel free to re-open a new issue/RFC to discuss the possible new syntax or create a PR.


.DS_Store still gets recognized regardless of core settings

You should make sure to enable the Hide Ignored Names and Hide VCS Ignored Files settings of the Atom core package tree-view.
The default values are false.

Ok thank you very much.
On Sun, Jun 11, 2017 at 08:23 Arctic Ice Studio<EMAIL_ADDRESS>wrote:

You should make sure to enable the Hide Ignored Names and Hide VCS
Ignored Files settings of the Atom core package tree-view
https://github.com/atom/tree-view.
The default values are false.
[image: ghi-14752-scrot-tree-view-package-settings]
https://user-images.githubusercontent.com/7836623/27012139-904d288e-4eca-11e7-8f93-077637467360.png
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/atom/atom/issues/14752#issuecomment-307635998, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AXXgslQ_TfSXnC2XEXxoMSaFQtM633YXks5sDAZrgaJpZM4N2Pk6
.

Looks like issue has been resolved.