UTF-8 letters replaced with null bytes if provided via Windows Terminal

I have an issue where running such code on Windows results in letters replaced with null bytes. Then it keeps failing in places like Path.windows() where are checks for null bytes - that's how I noticed it.
The issue appears only when running the script/.exe file from the terminal, both cmd and PowerShell.
Crystal
Code main.cr:
puts "Give me your name"
input = gets
value = input.not_nil!
puts value.inspect
puts value

While on Linux it worked perfectly
Linux (WSL):
$ crystal run main.cr
# => Give me your name
zażółć gęślą jaźń
# => "zażółć gęślą jaźń"
# => zażółć gęślą jaźń

Reading the value from terminal on Windows replaces some letters with null bytes
Windows 11
$ crystal run main.cr
# => Give me your name
zażółć gęślą jaźń
# => "za\u0000\u0000\u0000\u0000 g\u0000\u0000l\u0000 ja\u0000\u0000"
# => za gl ja

Windows Crystal Version:
$ crystal -v
# =>  Crystal 1.9.2 [1908c81] (2023-07-19)
# =>
# =>  LLVM: 16.0.3
# =>  Default target: x86_64-pc-windows-msvc

Based on the documentation, the default encoding should be set to UTF-8 and all the mentioned letters are valid UTF-8 letters.
If an encoding is not set, the default one is UTF-8.

When value is placed direcltly in the code like so:
value = "zażółć gęślą jaźń"
puts value.inspect
puts value

Windows 11:
Then there are no issues with printing values:
$ crystal run main.cr
# => "zażółć gęślą jaźń"
# => zażółć gęślą jaźń

Ruby
puts "Give me your name"
input = gets
value = input.chomp
puts value.inspect
puts value

Windows 11:
$ ruby main.rb
# => Give me your name
zażółć gęślą jaźń
# => "zażółć gęślą jaźń"
# => zażółć gęślą jaźń

We may refer to the solution in golang:
#https://github.com/golang/go/issues/4760
#https://github.com/golang/go/commit/26498684cb71de328b28974b87df3d5ab9486e4c

read_line is not that complicated:
require "winerror.cr"

lib LibC
  fun ReadConsoleW(hConsoleInput : HANDLE, lpBuffer : Void*, nNumberOfCharsToRead : DWORD, lpNumberOfCharsRead : DWORD*, pInputControl : Void*) : BOOL
end

def windows_console_read_line()
  buffer = uninitialized UInt16[32768]
  read_size : UInt32 = 0
  if LibC.ReadConsoleW(Crystal::System::FileDescriptor.windows_handle!(0), buffer, buffer.size, pointerof(read_size), nil)
    return String.from_utf16(buffer.to_slice[0, read_size]).strip # strip "\r\n"
  end
  raise WinError.value.formatted_message
end

p! windows_console_read_line() # it can handle unicode input

but for the custom delimiter it may be more complicated: https://stackoverflow.com/a/70124526

We're explicitly setting the console code page to UTF-8:
https://github.com/crystal-lang/crystal/blob/9390ee503b290e79c0dd886642cd1ea033d52b2e/src/crystal/system/win32/file_descriptor.cr#L279-L290
Apparently, that does not seem to work as one would expect?

#13750 should fix this.
I'm still unsure about the console codepage setting (#13731 (comment)). Is it really useful/necessary?

Hi, core team!
I'm a Windows user, and non-English user.
I've been following this issue, and I also found that there are already some solutions here( #11527). @HertzDevil
ReadConsoleW / WriteConsoleW is a better solution, and I see @straight-shoota  have created a PR. But there are some problems with this PR:

Incomplete characters in the buffer are not taken into account.
EOF is not taken into account.
Each time unbuffered_read is called, memory is allocated to the u16buffer.

This issue is a little more complicated than it seems. I think I can solve it better, but I need a few days to get used to Crystal.
If there is no hurry to fix this issue，can you give me a chance to contribute?

Sure, please give it a go 👍 Let me know if you need any help!

Sure, please give it a go 👍 Let me know if you need any help!
My implementation is only minimal to solve the primary problem. Of course it can be improved.
I suppose a potential optimization could be to use a StaticArray as u16 buffer and iterate multiple times if it does not fill the u8 slice at once.

Thank you, I need a few days (possibly more) to implement it because I am new to Crystal.

I've created a PR #13758

Thanks for the PR! I understand it's a lot of work doing that while still getting aquainted with the language. But I'm sure it also helps moving that forward =)
I would like to discuss some general design issues.

This is very complex code that needs to deal with a lot of edge cases. We need a way to write extensive tests. I'm not sure if we can sufficiently setup a console handle for testing, but I suppose we might use AllocConsole and then test read/write operations on that.
Can the value of GetConsoleMode change for a given file descriptor? If not, we should probably store the result instead of querying it on every operation.
I'd like to avoid buffering data in the file descriptor object. Typically, there's already a buffer in IO::Buffered and layering buffers doesn't do any good here. It's important to note that the slice that's being read into does not need to be filled to the brim. We can read from the console with sufficient margin to not overfill the slice. I believe UTF-8 encoding needs at most 50% more bytes than UTF-16 in case all codepoints are between U+0800 and U+FFFF (UTF-8 requires 8 bytes and UTF-16 only 2). In all other code ranges, UTF-8 is either smaller or equal in size.
We might not be able to avoid storing residual bytes, though. But that would be just 3 bytes max.
The documentation mentions that setting the console's code page may still be necessary to prevent other system config to produce garbage. But I'm not sure I understand it correctly.
A potential performance enhancement in the future (should not be part of the base PR!) could be to optimize for the common use case of IO#gets which looks for a delimiter (usually \n). I suppose we could provide a special implementation for that method using ReadConsoleW with pInputControl parameter which serves exactly the same purpose.

Thank you for your reply.
Yes, the code is complex, and I tried to explain the design idea:

I agree with you that this code is not easy to test. I only tested some cases by hand.
The value of GetConsoleMode may change, as mentioned in the documentation: windows/console/readconsole.
I know that the maximum number of UTF-16 code units that need to be read is no more than the number of bytes that need to be read, and I use this feature in my code. I used two buffers: 'console_bytes_buffer' and 'console_units_buffer'. Buffers are used for two reasons, one is to properly handle the residual UTF-16 code units and the residual UTF-8 bytes (For example, the character to be read has 4 bytes, but we only read 1 byte with 3 bytes remaining), and the other is to avoid allocating memory again and again.
I tested on the non UTF-8 code page, both Windows terminal and conhost, and it worked fine.
This code is already very complex, and I think it's best not to optimize it too early unless we really hit a performance bottleneck.

It seems impossible to get actual consoles for testing purposes from the Win32 API as there is ever maximal one console per process.
However, I figure we could mock the behaviour by subtyping IO::FileDescriptor with a stubbed implementation of the API methods. I've implemented a demonstration for read_console in this branch: https://github.com/straight-shoota/crystal/tree/test/windows-console-utf8-spec Do you think that's adequate for testing?
Right. If a descriptor gets redirected, it might not be a console anymore.
I don't think a persistent buffer for wide chars (UTF-16) should be necessary. IO::Buffered takes care of generic read buffering. We just need to read UTF-16 data from the console and put it into the buffer transcoded to UTF-8. If we never request more characters than can be put in the UTF-8 buffer, there's no need for persistence. So the only use case would be avoiding local allocations for performance reasons. I think it's best to disregard this for now and just focus on a simple implementation that is minimally invasive (i.e. does not add a bunch of instance variables to IO::FileDescriptor) and correct. We can optimize late and there are several options to explore.

It's a pleasure to communicate with you @straight-shoota

The only limitation is that it does not work well with miniscule buffer sizes, but that's not required because IO::Buffered always allocates a sufficiently siced buffer.

That's exactly what I care about. I'm not good at using English, so let's get straight to the code:
Example 1: Miniscule small buffer sizes are possible, and we cannot ignore this situation, and there will be residual UTF-8 bytes!
STDIN.read_buffering = false
slice = Bytes.new(1)
STDIN.read(slice)

Example 2: We cannot guarantee that the last code unit of the UTF-16 sequence read by ReadConsoleW is a complete character:
STDIN.read_buffering = false
slice = Bytes.new(3)
STDIN.read(slice)

# Then type "𐩯" in the console. NOTE: character '𐩯' has 4 UTF-8 bytes and 2 UTF-16 code units.

Example 3: This situation is not particularly important and can be achieved in other ways:
gets
gets

# Then type "A^ZB", `^Z` is `Ctrl-Z`, which stands for EOF.

My code is complicated because it takes into account several of the above scenarios.
Your idea of "re-using the buffer for reading UTF-16" is fantastic! We really don't need to console_units this way.
But I think it's indispensable to deal with the residual UTF-8 bytes and the residual UTF-16 code unit.

Example 1: Miniscule small buffer sizes are possible, and we cannot ignore this situation, and there will be residual UTF-8 bytes!

I'd argue that this use case might not be relevant enough that we should care about it. IMO IO::Buffered is intended to provide large buffer sizes and deviating from that can have defective effects.
Of course, this is up for debate. I'm happy to consider practical applications for that.

Example 2: We cannot guarantee that the last code unit of the UTF-16 sequence read by ReadConsoleW is a complete character:

I suppose this would happen when the buffer is filled fully and the end of the buffer is exactly inside a surrogate pair.
My solution for dealing with that would be to request one less UTF-16 unit from ReadConsoleW and do a second read of one unit in case the last unit is a surrogate.

String.each_utf16_char is currently private, and making it public requires careful consideration

I'm not making it part of the public API, see the :nodoc: addition. It only becomes callable from outside its scope. That's completely legitimate and applied in many similar situations. Don't worry about this.

I agree with you that we should not over-engineer for the sake of situations that theoretically exist but do not actually exist.
I reimplemented a version based on your suggestion. @straight-shoota

Regressions are regressions, the examples with read buffering disabled should not break. It should suffice to use a local StaticArray for very small output slices without introducing a lot of code changes

I suppose we could introduce a special case handling for buffer.size < 6 (I think that's the lower limit of the current implementation). It should be as simple as using a small static array for the units buffer if the buffer size is too small.

Moreover I am not sure if IO::Buffered#read is allowed to touch the parts of the output slice that do not correspond to the bytes successfully read. If STDIN can assume those bytes aren't used by the caller, the caller can also assume stdlib doesn't use those bytes on equal grounds.

I would not worry about that. IMO a buffer supplied to a method can be used to the full extend of its capacity for whatever utility purpose. Regardless of how much filled space is reported as a result. This is very pracitcal and can be clearly documented.
Furthermore, I don't see much practical use for assuming that buffer space beyond the number of read bytes is untouched. What kind of purpose would that enable?
I wouldn't even be sure if the invariant isn't broken in existing implementations already.

It should suffice to use a local StaticArray for very small output slices without introducing a lot of code changes.

I'm not sure I didn't understand some information.
PR #13758 has been trying to properly handle the 2 problems:

There is no guarantee that the data read by the ReadConsole is full Unicode characters.
There is no guarantee that the data fill into the slice is full UTF-8 bytes.

What is now certain is that the first problem can be solved with two reads. And there has been no perfect solution to the second problem:

The initial approach handled both problems correctly, but increased the size of the file descriptor.
In the ensuing discussion, I proposed a way to improve, but it was rejected.
The current code is relatively simple and can handle the general case, but it cannot handle the case where the slice is too small.

I haven't figured out how to solve the second problem with just a local StaticArray.

Moreover I am not sure if IO::Buffered#read is allowed to touch the parts of the output slice that do not correspond to the bytes successfully read. If STDIN can assume those bytes aren't used by the caller, the caller can also assume stdlib doesn't use those bytes on equal grounds.

In other programming languages, similar approaches are also used (such as golang).
Usually, this is not a problem. One problem that comes to my mind is that the caller assumes that the remaining part of the slice are the initial values (such as zero values), but this approach breaks this assumption.
We need to make a reminder in the document, for example, the following reminder comes from golang standard library document:

Even if Read returns n < len(p), it may use all of p as scratch space during the call.

It should suffice to use a local StaticArray for very small output slices without introducing a lot of code changes.

I'm not sure I didn't understand some information.

It's not about edge cases in the data, but the size of the provided buffer. My comment refers to the first paragraph of https://github.com/crystal-lang/crystal/issues/13731#issuecomment-1745613302, which goes along with the example in https://github.com/crystal-lang/crystal/pull/13758#discussion_r1344633947.
A buffer size of 4 is enough to hold any single UTF-8 codepoint, so it should technically work. But the current implementation raises on third < 2, which implies buffer.size < 6.
It should be totally fine to handle this tiny buffer as an edge case.
The proposed solution is to use a StaticArray(UInt16, 2) as units buffer if buffer.size < 6.

There are still two points that we have not reached an agreement on:

For units_Buffer, whether to use the remaining of slice or a local StaticArray. I tend to use one of the methods instead of mixing the two. If using StaticArray, what is the appropriate size? 64 or 128?
How to handle the second problem? We have already attempted to solve it (the initial solution) and have also attempted to ignore it (the current solution). How can we balance simplicity and completeness?

For units_buffer, whether to use the remaining of slice or a local StaticArray. I tend to use one of the methods instead of mixing the two. If using StaticArray, what is the appropriate size? 64 or 128?

If we agree that mutating the buffer beyond the returned data is acceptable (@HertzDevil WDYT?), we should use slice as units_buffer. Only if slice is very small, we must use a static array. There is no problem with mixing both approaches.

There is no guarantee that the data fill into the slice is full UTF-8 bytes.

I don't think there needs to be a strong guarantee for that. With a couple of constraints in place, this edge case should be pretty negligible. For example, the slice should always be large enough to hold a full UTF-8 codepoint. If slice.size < 4, undefined behaviour should be acceptable.

STDIN.read_buffering = false
STDIN.read_byte

relies on reading into a 1-byte Slice, so it must not be undefined behavior.

Are we talking about an actual use case or just a theoretical scenario? What's the point in disabling read buffering and reading individual bytes? I don't see a practical use case for that.
We should be careful about complicating the implementation for hypothetical edge cases.

We are talking about never introducing any regressions, however minor. So if the internal decoder cannot be implemented without buffering, then maybe the original idea of having a separate read buffer isn't that bad after all.

maybe the original idea of having a separate read buffer isn't that bad after all.

The most minimal step for implementing the edge case of reading slices that are smaller than a UTF-8 codepoint is a small buffer for holding at most 3 residual bytes.

I have implemented two versions, one that completely handles edge case correctly, and the other that ignores edge case. But we have never reached a consensus on how to handle edge case.
I am personally a pragmatist who values simplicity over completeness. So I agree with the current implementation.
Of course, I don't object to correctly handling edge case either. I just need to revert to the original code of this PR and make some small improvements to avoid increasing the size of the file descriptor.

The most minimal step for implementing the edge case of reading slices that are smaller than a UTF-8 codepoint is a small buffer for holding at most 3 residual bytes.

It is too early to discuss this detail now, as we have not yet reached an agreement on edge case handling. Ultimately, whether to use a 3 bytes buffer or an appropriately sized buffer depends on simplicity and performance.

maybe the original idea of having a separate read buffer isn't that bad after all.

Do you agree to use a separate read buffer while keeping the code simple? @straight-shoota
The initial code was indeed a bit complex, but we can use the discussion results of this period to simplify it.

Yeah, I suppose we can do that. Let's get it working in a state that we all agree is sane. We can always optimize the implementation later on.
I think we only need an ivar holding a utf-8 buffer in IO::FileDescriptor. By default, this would be empty. It allocates when read_console is called.

I have refactored the code, and the current code can handle edge case correctly without adding much complexity. @straight-shoota @HertzDevil

By using the class variables, the problem of increasing the size of file descriptors is avoided.
Not using the remaining portion of slice, because there are still different opinions on this approach.
Did not use StaticArray or 3-bytes buffer, but instead used a regular sized buffer because it was simpler.
No lazy allocating memory because the buffer is very small and only allocated once during program startup.


Please add a "--leader" switch to the "flynn route" command

Hi there!
I really like the way Flynn comes with data replication "from the moment you turn the key".
However, I'd like to be able to connect to the database remotely, so that I can set up an off-site postgresql instance as a streaming replica.
I asked in the #flynn IRC channel about my needs, and whilst I can add a route with "flynn -a postgres route add tcp -s postgres-postgres -p 5432", it looks like it isn't possible to configure the route to always send traffic to the leader.
Would it be possible for you to add a "--leader" switch to the "flynn route" command, to route traffic to the service leader only?
Thanks in advance for considering my feature request.
Kind Regards,
Hazel Smith

:+1: this opens up great possibilities for data-layer migration, which is going to be a need for us soon.

Thanks!


Use travis API v3 on status.codeforiati.org

Travis API v3 seems like a big improvement on previous versions. See:
https://api.travis-ci.com/v3/
Combining this with a JSON pytest reporter ought to make it possible to show detailed test results on the dashboard.

I decided not to do this… Instead:

pytest spits out a JSON report, which gets auto-pushed to github pages
jekyll generates a static site using the JSON


Checkpoint to phase 2 of the great ustringhash conversion

Reminder: phase 1 changed the RendererServices APIs to be based completely on ustringhash for public method calls, not ustring.
This PR starts along the path of phase 2.
In oslconfig.h, I define a new type ustringrep which is the in-memory/in-shader CPU-side representation of a ustring. Currently, it's still equivalent to a ustring.
I also define a type called ustring_pod, which is the "plain old data" representation inside a ustringrep (i.e., a const char* when it's a ustring, and a uint64_t when it's a ustringhash). For reasons not important, it's exceedingly hard to pass a true ustring/ustringhash as parameters to or from LLVM-jitted code without running afoul of LLVM's type checking, so it's just easier to pass the underlying data type.
So the main work of phase 2 is to change everything related to shader data from a ustring into a ustringrep, and things passed as const char* into passing a ustring_pod. In some sense, it's just renaming and type conversion plumbing.  When that's done, phase 3 is to throw the switch and change the definition of ustringrep to ustringhash (and of course fix everything that's broken).
This PR does not complete phase 2, but takes a big bite out of it. It's an arbitrary bite, not much rhyme or reason to which specific parts I did so far. Just working on it piece by piece and felt that this was a place where much has been done, everything still works, and it's a reviewable chunk without getting too much out of control.
Areas I worked on are related to range_check, raytype, getattribute, bind_interpolated_param, uninit_check, naninf_check, getmessage/setmessage, some texture setup, pointcloud.
Signed-off-by: Larry Gritz<EMAIL_ADDRESS>

When all this is totally done (past stage 3 with the switch thrown), would the idea be to remove ustringrep and just have ustringhash?

Consider changing name of ustring_pod to ustring_param as that makes it more clear where its going to be used.

Ha, it was ustring_param until the last minute, when I thought that was unclear and renamed it ustring_pod. I'm still on the fence about the final name. I think where my head was can be summarized as that "param" made me think "how is this different from passing a ustring as the param?" whereas the pod was to emphasize that it's the non-class, plain-old-data representation of what's inside a ustring.

When all this is totally done (past stage 3 with the switch thrown), would the idea be to remove ustringrep and just have ustringhash?

Yeah, I think so. I wanted a temporary name in the interim that I can easy switch back and forth. Once everything is working, it may just get wholesale replaced by ustringhash.

I think we should treat all this nomenclature as temporary. When this work gets completed, some of them will no longer be necessary at all, and for others we can then choose what names we want on a permanent basis.

it was ustring_param until the last minute, when I thought that was unclear and renamed it ustring_pod. I'm still on the fence about the final name. I think where my head was can be summarized as that "param" made me think "how is this different from passing a ustring as the param?" whereas the pod was to emphasize that it's the non-class, plain-old-data representation of what's inside a ustring.

I had similar issue, mostly when do I use ustring_pod vs. ustring vs ustringhash

I think it just might be a bit weird because its in an intermediate state.  Forward looking,
ustringrep would just be gone, replaced with ustringhash
ustring_pod would be gone, replaced with ustringhash::hash_t

Yeah, I think something like that is the final state.

Maybe don't worry about the names then (as they are intermediate) but I do recommend killing off the casting USTR macros and using helper functions.

Ok, I'm going to merge as it is and move on, cleaning up these things we've mentioned as I go.


VDB Sequence seems to be skipping or remapping frames?

4 frames ahead @ frame 19 and 14 frames ahead @ frame 60

I've forwarded this to my colleague Sami who implemented the Sequencer support.
We need more time and exploration to understand what is happening. Will let you know when we have an update, but it may take a while.

I ran into this issue last night as well. I was debugging it this morning before I found this issue and noticed if I set the Sequencer FPS  to 29.97 it will not skip any frames. Hopefully that's helpful info.

I've taken the time to look at it  and study it, and this is just the "normal" behavior. Depending on your chosen FPS in the sequencer, the frames will adapt to reflect the FPS.
Like vgrant says, the "default" value should be 30FPS, or 29.97FPS to be even better.
Hope this helps. I will close this issue as it is not an issue in the end.


The type or namespace name 'System' could not be found

Hey,
I'm trying to open ILCompiler.sln but I'm unable to compile any projects there, I tried both with Visual Studio 2017 and 2019

I tried to reclone entirely corert, init-tools.cmd but it still doesn't work.
From the screenshot above I tried to compile repro.csproj and the command line from msbuild is like this:
C:\code\dotnet\corert\packages\microsoft.net.compilers.toolset\3.3.0-beta2-19367-02\tasks\net472\csc.exe /noconfig /unsafe+ /nowarn:169,414,1701,1702,2008 /nostdlib+ /warn:4 /define:X86;BIT32;PLATFORM_WINDOWS;CORERT;DEBUG;TRACE;DEBUGRESOURCES /errorendlocation /preferreduilang:en-US /debug+ /debug:portable /optimize- /out:C:\code\dotnet\corert\bin/obj/Windows_NT.x86.Debug/repro\repro.exe /ruleset:"C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Team Tools\Static Analysis Tools\\Rule Sets\MinimumRecommendedRules.ruleset" /target:exe /warnaserror+ /utf8output /checksumalgorithm:SHA256 /langversion:preview /analyzerconfig:C:\code\dotnet\corert\.editorconfig Program.cs

So it is passing /nostdlib+ but I don't see any netstandard library passed to it for example, so not sure how this can work.
Any clues?

Run build.cmd to build and restore everything. This will build x64 debug by default. Then open in VS and set the same config as you just built (do not leave it at AnyCPU). That should resolve it.

Ok, it seems an issue with Resharper (and Rider is not working as well), super annoying... but closing the issue


Can't apply patch

Tried to create a PR for a patch generated by nodejs patch-package module. This tool checks out a clean copy of the module to be patched and diffs it with the changes in the modules node_modules folder hirarchy. The resulting patch is saved to patches/module.name.patch
I used this patch to run patch2pr and got this error:
patch2pr -repository mrousavy/react-native-vision-camera patches/react-native-vision-camera+2.15.4.patch
error: apply failed: node_modules/react-native-vision-camera/android/src/main/java/com/mrousavy/camera/CameraViewModule.kt: no entry for modified file

Is it possible to share the patch you created that failed to apply? That will make it easier to debug this problem.
Based on your description and the error, I think the problem is that patch-package creates patches that are meant to apply to installed copies of libraries and not to the upstream source repositories of the libraries.
From the output, you can see that it's trying to modify a file in the node_modules/react-native-vision-camera directory, but this directory only exists after you run npm install react-native-vision-camera in a project. If you look at the upstream repository, the android directory exists at the root and there's no node_modules directory.
You might be able to fix this by pre-processing the patch to change the paths, maybe something like:
sed 's|mrousavy/react-native-vision-camera/||g' patches/react-native-vision-camera+2.15.4.patch | patch2pr -repository mrousavy/react-native-vision-camera

But in general, installing an npm package extracts an archive that is derived (maybe via a build process) from the upstream source repository, so there's no guarantee that the files you have in your node_modules directory exist in the source.
A final note: patch2pr requires write permissions on the repository to create a PR, so you'll need to create a fork and target your fork with the -repository flag if you are not a contributor on the original repository.

Ah thanks that makes so much more sense now! Yes the patch is generated upon the distributed set of files, not the actual source files. That explains it.
Also I missed the part that I need to create a fork first and stream my PR into that, but I should have known, since that is the default PR flow anway.
Thanks I will close this and try the sed solution.


Random errors with Stash\Utilities::deleteRecursive()

Actually had this with our behat test suite for the last year or two, it's random so haven't been able to pinpoint what the issue is. And re-reading the Stash code now I still can't quite make out what happens.
But my suggestion is that we simply add dependency on symfony/filesystem and use that instead in there to take care about recursive deletion, it's well tests on a range of file systems so there is good chance it will be able to be more solid for this.
Error we are getting is always in this form:
Warning: rmdir(app/cache/behat/stash/0fea6a13c52b4d47/25368f24b045ca84/c21f969b5f03d33d/43e04f8f136e7682/4a1c6be177996f9e/03934ae1c1c02ffc): Directory not empty in vendor/tedivm/stash/src/Stash/Utilities.php line 172 in vendor/behat/behat/src/Behat/Testwork/Call/Handler/RuntimeCallHandler.php:85
  ╳  Stack trace:
  ╳  #0 [internal function]: Behat\Testwork\Call\Handler\RuntimeCallHandler->handleError(2, 'rmdir(...', 'vendor...', 172, Array)
  ╳  #1 vendor/tedivm/stash/src/Stash/Utilities.php(172): rmdir('app/ca...')
  ╳  #2 vendor/tedivm/stash/src/Stash/Driver/FileSystem.php(335): Stash\Utilities::deleteRecursive(Object(SplFileInfo), true)
  ╳  #3 vendor/tedivm/stash/src/Stash/Driver/Composite.php(163): Stash\Driver\FileSystem->clear(Array)

This was with v0.14.2, but we have had this issue for a while so I don't think any changes in v0.14.x is to blame.

This may be a race condition- one process starts deleting and clears out the directory but another process adds new items in before that first process finishes.
If that is the case (and I strongly believe it is) then a simple fix that would work immediately would be to rename the root file before beginning the cleanup operation. That way other processes can immediately write to the cache without causing problems.

That being said I'm not opposed to switching to a more well polished solution such as the symfony/filesystem library, and it would be interesting to see if they already had a solution to this type of problem.

It appears the Symfony implementation combines error suppression operators with a second existence check, so it looks like this for a file delete for example:
$success = (@unlink($path) && file_exists($path));

Confirm seeing the same issue randomly when using the Filesystem driver.
A cursory look at the code does indeed suggest that this is a race condition.
As much as I would like to trust other filesystem access libraries, I think that using rename-first instead of a double-check on file/directory suppression is a much better option. Renames are more or less the only FS option that is guaranteed to be atomic, and are widely used by libraries that have to deal with high concurrency writes (eg. file writes are always write-file-with-tmp-name-then-rename). Having said that: hod does the system cope with renaming a directory while a file is being added to it?


feat: Added brain.fm support

I've added super crude support for brain.fm. There is no api nor embedded version. As such it may not comply with the aesthetics standart of Astrostation. Apart from that its functional and just requires user to login.

added to SideNav
added to WidgetControl
added brain.fm icon
default pos same as spotify

This is my first contribution to any open source project :)

You can get 30 days trial here -> https://my.brain.fm/payment?extended_promo=30&utm_source=referafriend :)

I dont think this is going to be something we want as it requires an account.
Some users are not going to want to create an account on this, and i dont want to add login requirements for astrostation

yeah, brain.fm could be bit of a niche service. still the optional (if you want to use it) onetime login just to brainfm seemed like a ok step as it remembers your authentication.
But I understand if you want to keep any login requiring widgets out.
Does that mean that the embeded spotify player requires no authentication and is a 'free spotify' player?

yeah, brain.fm could be bit of a niche service. still the optional (if you want to use it) onetime login just to brainfm seems like an ok step as it remembers your authentication.
But I understand if you want to keep any login requiring widgets out.
Does that mean that the embeded spotify player requires no authentication and is a 'free spotify' player?

Yeah - it was initially implemented to have just enough that a user can listen freely.
But im not sure if thats still the case - or how well it even works at this point


WIP: Refactors most of the current api endpoints

into a few different blueprints, and registers them on the app.
Spot-checked the majority of endpoints so should be OK to use, need to deploy to server and put all the endpoints into the tool to make sure it all actually still works.

@jasonrhaas Heads up, I went ahead and did this refactoring. Still need to fold in the declarative_app, which will allow us to delete the endpoint we currently have called list_all() (which I've left in application.py with a TODO to delete once done) .
Next up I'm going to go ahead and shift some of our tables to use the new zonefacts table and add in stuff to the filter endpoint, which are the big things our front end needs, as I sort through what exactly we currently need on the front end. We'll end up dropping a few outdated endpoints that we can also swap out on the front end if we do it that way, so we don't need to worry about testing all the existing ones exactly until that's sorted out.

@NealHumphrey I just get this when running python application docker.  Is this expected?
INFO:root:['meta', 'manifest', 'building_permits', 'subsidy', 'project', 'reac_score', 'real_property', 'dc_tax', 'zone_facts', 'topa', 'census', 'census_tract_to_neighborhood_cluster', 'census_tract_to_ward', 'crime', 'wmata_dist', 'wmata_info', 'zone_housingunit_bedrm_count']

@jasonrhaas I am pretty sure that comes from this line (line 90):
logging.info(tables)
Which is just echoing back the tables that it found when it connected to the database. Assuming so, yea that's expected (and was in there pre-refactor too).

@NealHumphrey With my latest commit I added the Flask Restless Blueprints to application.py.  I added them into the application itself since it relies on an actual application instance and connection to the database to create the models.
Let me know what you think.

@jasonrhaas This looks good. I agree putting the automap on application makes sense - and when we want to use the mapped models in any of our blueprints, we can pass them through via the blueprint constructor.
As a side note, I find it safer to use the 'one branch per person' workflow - i.e., rather than pushing directly to the 400-move-to-blueprints branch that I made initially, you make another branch off of that branch, and then do a PR either into the 400- branch, or replace this pull request by making a PR into dev. It's slightly more branch management, but with a team of diverse Git skill level it's one less thing to think about - you can always push to a branch that you made without thinking about whether there are any upstream changes. Just something to keep in mind for future development.
Merging in; I'm going to tweak a few things like pagination defaults after merging but will do that in another PR as I'm going to do the front-end changes to use the new endpoints concurrently so I can get them all working smoothly together.


[Question] Is there support for clang compilation database?

Hello :)
I'm kinda new to sharpmake and I couldn't find anything in regarding this - I was wondering if there is support for clang's compile_commands.json or something similar.
I'm hoping to use clangd as LSP, and I'd love to use Sharpmake to manage my project.
Is there currently a way to generate that?
If not, could you point me towards the bit of code that would be a good spot to inspect and hopefully be able to implement that feature?
Thank you in advance!

See this sample: https://github.com/ubisoft/Sharpmake/tree/main/samples/CompileCommandDatabase


Added control for item selections using checkboxes in explorer

Hi,
please commit if you find it usefull.
Thank you
Tom

Hello @Tikus,
I'm currently running the script as a 2 part login script in a GPO:

All mods that need admin rights, are run as a computer startup script
The rest of the mods, that affect the user, are run as a user login script

The second part is run as a user login script as if run as a computer startup script it would run as the administrator user, so it won't affect the logged in user.
Do you have experience when running this modification as a computer startup script? Will it modify the logged in user hive or will it continue modifying the admin one?
It would be great if this allows the script to be run all at once instead of having to split it.

Hi there,
Im using this script for 2 scenarios:

As part of MDT task sequence that setups defaults for the computer
Its used with -useDefaultUser option. In this mode it writes to c:\users\Default\ntuser.dat so any user created afterwards will inherit those settings automatically.

As a script that i execute under real users that tunes the rest (settings are different for some users)
In this mode HKCU is mapped to HKEY_USERS$currentUserSid so settings dont apply to admin but logged in user.

So yes you can run this script only once. Another benefit is that the user doesnt have to be in admin group.
Let me know if it works for you

When running the script as a computer start script or a scheduled task gpo (at the computer tree) with the user to run as SYSTEM, the $env:USERNAME returns the PC itself (eg TESTPC$) so it can't find the user sid and stays null.
How do you run it?
Thanks!

I alway run the script as local administrator. If u run it as system user on system where no local user exists  you could try adding -useDefaultUser argument as it would write to registry of default user so any local user created afterwards will inherit those settings. Script expects that there is some user logged in already.

The script is run on a system with a domain user that has already logged-in in the past, and the script is run as a user login script (through the task scheduler) for that user but executed as the SYSTEM user for admin rights.
It should run after the user logs in, but it doesn't seem to be able to get the username variable of the user instead of the computer.
I can't make it work, so I think I will resort back to splitting the presets and run them separate on a user or computer GPO which worked correctly.
Thank you for your time @Tikus

Since you're trying to merge your master instead of creating feature branches for PR, I can't pull this PR anymore, so I'll close it. Nevertheless, when I'll find some time, I'll skim through your changes as some of them look really interesting. Thanks.

Sry, gotta learn git well first :( Ill be back :)


Add S3 to block adapter unit tests

Linked Issue
Closes #6085

Added comments
Looks like we are doing integration testing as we spin minio to test the adapter

We are using a minio container to simulate S3 as I couldn't find some other S3 emulator. This is similar to what we do with Azure and Azurite


Hetzner dyndns console is missing after update to OPNsense 24.1_1-amd64 FreeBSD 13.2-RELEASE-p9 OpenSSL 3.0.12

Important notices
Before you add a new report, we ask you kindly to acknowledge the following:

[ x] I have read the contributing guide lines at https://github.com/opnsense/plugins/blob/master/CONTRIBUTING.md
[ x] I have searched the existing issues, open and closed, and I'm convinced that mine is new.
[x ] When the request is meant for an existing plugin, I've added its name to the title.

Is your feature request related to a problem? Please describe.
I just updated to OPNsense 24.1_1-amd64 FreeBSD 13.2-RELEASE-p9 OpenSSL 3.0.12 and the hetzner dyndns client is missing in ddclient configuration.
Anybody know how to get it back?
Thanks
Describe the solution you'd like
Using Hetzner dyndns console again. Can I revert to the point it is available again?
Describe alternatives you've considered
There is no
Additional context
Add any other context or screenshots about the feature request here.

Switch to ddclient backend in plugin settings?

using ddclient already, not legacy.
new: ddclient | 3.11.2_1 | 340KiB | OPNsense | GPLv2+ | Update dynamic DNS entries
old: ddclient | 3.11.2_1 | 340KiB | OPNsense | GPLv2+ | Update dynamic DNS entries
same version number, just dropped configuration support

thats the solution, thanks


Business listing request

Hi there,
My company makes an email client called Hiri. We now accepts Monero! I would appreciate it if you could list us on the following page:
https://getmonero.org/community/merchants/
You can buy Hiri using Monero here:
https://www.hiri.com/crypto/currency/XMR/
Cheers!
Dave
Founder
@hiri

Hi. Added your service in #645
+merchant
+in progress


[WIP] Add "duration" option for token command.

This addresses #63
It would make users happy who feel like the 15 mins is too short :)

@mattlandis @mattmoyer
Thank you for reviewing my PR!
I misunderstood that the 15 mins limitation just comes from the duration for the AssumeRoleProvider, but seems to depend on the limitation for signed request as well. (Sorry I had not tested this PR before created.. :bow:) So I think this PR should be closed and the design should be reconsidered. What do you think?

This is VERY annoying when using eks with iam roles and logging into k8 dashboard.
Literally every 15min one would need to re-login.

Yeah, we use the command below to give users a token to log into the kube dashboard with a token. Works great minus that every 15 minutes their browser session closes on them and they have to get a new token.  For kubectl people don't notice that sometimes a command takes longer, but they sure do notice in the web ui when it says session expired.
heptio-authenticator-aws token -i clustername | jq .status.token -r

Are there any news? It's really very annoying.


0x5E76AD3e1B2067aB88233B37942d64B75A520DDa

https://metamask.app.link/send/0x5E76AD3e1B2067aB88233B37942d64B75A520DDa

Have anything. U don t really want!!!


display:grid can't support

when use display:grid API in CSS3, corsswalk donesn't support now.
corssWalk version is https://download.01.org/crosswalk/releases/crosswalk/android/maven2/org/xwalk/xwalk_core_library/23.53.589.4/

Crosswalk is no longer being developed so it's unlikely to support CSS Grid


External ruleset not picked up by super linter

Describe the bug
I am trying to reference external ruleset in my swagger.yml in GitHub action using the http url as per the doc:
https://docs.stoplight.io/docs/spectral/83527ef2dd8c0-extending-rulesets
But it wont work.
Expected behavior
workflow should recognize and pick up external rule file , given the http url of the rule file
Steps to Reproduce
Refer external linter rule file with http url using GitHub actions with Super Linter as per stoplight spectral documentation

@jayanthpatki91 can you please share more information about how you're running super-linter in your repository?


Update openEuler image tag

Has the dependent PR merged yet?

Merge this after https://github.com/opencv-infrastructure/opencv-gha-dockerfile/pull/28/files#diff-d139e503934c4264e7462305bdd2e0adf873ca72ce783b24ba3f61ebfea67447R1

Error response from daemon: manifest for quay.io/opencv-ci/opencv-openeuler-22.03.sp2-ascend310:20240220 not found

@asmorkalov Could you trigger a build on the new image? I just merged https://github.com/opencv-infrastructure/opencv-gha-dockerfile/pull/28.

@fengyuentau I built the image. CI passed.


Modernize type hints

Convert from Python2 style comment type hints to true Python 3 annotations.
This was done with com2ann.
This PR only covers those files that have not been edited recently, and is split by top-level directory to reduce the chance and cost of conflicts.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

R: @tvalentyn

Run Python 3.8 PostCommit

postcommits will be here: https://github.com/apache/beam/actions/runs/9848030460
there were linter failures as well - have you also tried importing this and running internal tests?

Good call. Internal tests look good on import.

Fixed the lint issue.

Yes, looking at the manual changes and spot checking the auto generated should be sufficient.
The worst thing is failure to import because of some circular/forward definition. I think I fixed all of those (even if things aren't imported by default, the docbuilder import most modules.) As you mention, some coders are inferred from type hints, but that should fail quickly because of actually being incorrect (and I didn't run into any of this in all the tests).

Also, if there are issues we should be able to revert individual commits rather than roll the whole thing back.
I'll wait 'till the release is cut before merging this to give it extra time to bake.


fix(ssh_tunnel): Fix bug on database edition for databases with ssh tunnels

SUMMARY

If the SSH_TUNNELING feature flag is OFF no SSH Tunnel Switch or the rest of the form fields must be visible in the DatabaseModal, however, that's not happening when editing a DB. Add test that check that if no switch is in the document, then the rest of fields must not be in the document.

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF
Before:

After:

TESTING INSTRUCTIONS

Make sure your SSH_TUNNELING flag is OFF
Open the edit modal on a DB that has SSH Tunnel attached to it
The SSH Tunnel switch must not be visible
The rest of SSH Tunnel form fields must not be visible

ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Required feature flags:
[ ] Changes UI
[ ] Includes DB Migration (follow approval process in SIP-59)

[ ] Migration is atomic, supports rollback & is backwards-compatible
[ ] Confirm DB migration upgrade and downgrade tested
[ ] Runtime estimates and downtime expectations provided

[ ] Introduces new feature or API
[ ] Removes existing feature or API

I left a comment on a non-blocking nit, otherwise this looks great!

Thanks! PR updated with the suggestion 🙌


PIP | Conda | Etc. Installation

Above is a great convince to have.

@sirinath I've added installation instructions to the README. I took a look at conda and since prophet is pure python, I think I'll leave it as a pip package. People can still use conda to manage packages with C extensions but use pip to install prophet.
Doing conda packaging properly using cross platform packaging introduces overhead I don't want to manage currently as I would need release build machines for every supported OS. Because it's pure python, I could release an OSX version and people can download and manually convert the package for their OS and it should work, but the package not found errors for specific OS's will confuse people new to conda.


⚠ Bofrost.at has degraded performance

In be12683, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1690 ms

Resolved: Bofrost.at performance has improved in 7622b7d.


unit tests

use pytest to perform unit tests.

[x] preprocess
[x] SVD
[x] map to map
[x] distribution to distribution
[ ] testing notebooks (low priority). See here and here

see example of pytest the compSPI/simSPI repo https://github.com/compSPI/simSPI/blob/master/.github/workflows/test.yml
Suggestions

perform tests on subset of ground truth and submission data (e.g. 100 gt maps, 10 submitted maps), so that test runs in 10s - 1 min.
make use of data validators to pass/fail test

For testing the notebooks, it would be nice to have a test config and pass in paths (overwrite the interactive aspect).

Closing - continue discussion about testing plots in this issue: https://github.com/flatironinstitute/Cryo-EM-Heterogeneity-Challenge-1/issues/42


Since 2.2.0: Error: Please provide Nexmo API credentials

Hello!
Since, 2.2.0, I receive the following error:

Please provide Nexmo API credentials. Possible combinations: api_key + api_secret, api_key + signature_secret, private_key + application_id, api_key + api_secret + private_key + application_id, api_key + signature_secret + private_key + application_id {"exception":"[object] (RuntimeException(code: 0): Please provide Nexmo API credentials.

I reverted back to 2.1 and all is ok. Using Laravel 5.8.
In config/services.php I have the Nexmo credentials:
    'nexmo' => [
        'key' => env('NEXMO_KEY', '<my key>'),
        'secret' => env('NEXMO_SECRET', '<my secret>'),
        'sms_from' => '<my phone>',
    ],

@vladrusu you're not setting the credentials directly in your config file I hope? You should set them using the env variables.

Hi !

@vladrusu you're not setting the credentials directly in your config file I hope? You should set them using the env variables.
Yes I do. I set them on .env only on my dev machine. On the production server, if anyone has read access to the php config file, has also access to the .env file. If I am wrong from a security POV, please correct me.

Nevertheless, here is my code:
App\Notifications\SMS.php
<?php

namespace App\Notifications;

use Illuminate\Notifications\Notification;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Bus\Queueable;
use Illuminate\Notifications\Messages\NexmoMessage;

class SMS extends Notification implements ShouldQueue
{
	use Queueable;

	public $content;

	public function __construct($content)
	{
		$this->content = $content;
	}

	public function via($user)
	{
		return ['nexmo'];
	}

	public function toNexmo($user)
	{
		$o = (new NexmoMessage)->content($this->content);

		return $o;
	}
}

I call the notification with:
$user->notify((new \App\Notifications\SMS('Example SMS text')));

Yes I do. I set them on .env only on my dev machine. On the production server, if anyone has read access to the php config file, has also access to the .env file. If I am wrong from a security POV, please correct me.

I indeed think this is unwise as your config files usually get committed with your source code (which is shared with other developers maybe) and your env file resides on your (secure) server.
This probably happened because of this PR which removed the keys from the config but keeps BC with the naming of the environment variables: https://github.com/laravel/nexmo-notification-channel/pull/22
So I suggest you switch to only using environment variables for this (or publish the nexmo config but I don't recommend doing that).


feat: support stamp substituions

Type of change

New feature or functionality (change which adds functionality) #431

Test plan

New test cases added

made sure that failures are isolated to missing toolchain hashes/artifacts for expand_template.

Lets create a new expand_template.bzl load point for this rule with the PR and a new docs file for that so it shows up separately in the docsite. The expand_make_vars load point can stick around for now for backward compat with a # TODO(2.0): remove comment on it.

Lets create a new expand_template.bzl load point for this rule with the PR and a new docs file for that so it shows up separately in the docsite. The expand_make_vars load point can stick around for now for backward compat with a # TODO(2.0): remove comment on it.

done


Determine collator reference hardware

Q: Are collators expected to use Polkadot's own reference hardware?
A: Yes, see #354, we are going with the recommended configuration, which uses x86

Either way it needs to be documented and then used to generate new weights

[x] @shannonwells to run the current weights on the c6id.2xlarge to see what the numbers currently look like.

[x] Messages
[x] Schema
[x] MSA
[x] ORML Vesting
[x] Scheduler

I propose we just use the Polkadot Validator requirements (most common to use across other parachains as well)
https://wiki.polkadot.network/docs/maintain-guides-how-to-validate-polkadot/#requirements

We haven't yet; I'm about to check in some weights generated on the Polkadot reference hardware for helping make that decision.

Chosen Reference Hardware (Copied from Polkadot Validator Reference hardware on 2022-08-23)
CPU
    x86-64 compatible;
    Intel Ice Lake, or newer (Xeon or Core series); AMD Zen3, or newer (EPYC or Ryzen);
    4 physical cores @ 3.4GHz;
    Simultaneous multithreading disabled (Hyper-Threading on Intel, SMT on AMD);
    Prefer single-threaded performance over higher cores count. A comparison of single-threaded performance can be found [here](https://www.cpubenchmark.net/singleThread.html).
Storage
    An NVMe SSD of 1 TB (As it should be reasonably sized to deal with blockchain growth). An estimation of current chain snapshot sizes can be found [here](https://paranodes.io/DBSize). In general, the latency is more important than the throughput.
Memory
    16GB DDR4 ECC.
System
    Linux Kernel 5.16 or newer.
Network
    The minimum symmetric networking speed is set to 500 Mbit/s (= 62.5 MB/s). This is required to support a large number of parachains and allow for proper congestion control in busy network situations.

Chosen versions on Common Cloud Platforms

Google Cloud Platform (GCP): c2d-highcpu-8
Amazon Web Services (AWS): c6id.2xlarge

@wilwade RocksDB tries to do sequential writes to storage. So it's optimized for HDD and not SSD. I think there are some rocksdb flags that optimizes it for SSD. Based on this it's hard to tell if we should use HDD or SSD. Maybe we can ask Parity.

Parity says they have done a lot of optimization around RocksDb, so we likely shouldn't have to do any more.


Gogs pops up "Do you want to leave this site" on login page

I upgraded from v0.9.13 to v0.9.46 today and immediately noticed a new bug which appears to be due to the changes done for issue #2881.
If I go to my gogs login page then chrome automatically pre-fills my name etc into the login form just like any other site login page. If I immediately try to close the tab, without typing anything, then gogs pops up a "Do you want to leave this site. Changes you made may not be saved" window. This is annoying to the user and no other login page does it.

Patch has pushed to fix this issue, please test on master branch or https://try.gogs.io.


In BlogDataStore.IteratePosts there is no need to take a list and return it

We should do one of the following:

Don't take a list, but create one and return it
Accept a list, fill it in the method, but don't return it since the caller already has reference to it

Fixed in PR #174


Add the altitude at which the rocket leaves the dimension

Forge Version
<IP_ADDRESS>54
Galacticraft Version
4.0.281
Log or Crash Report
N/A
Reproduction steps
My English is not very good, so this is translated by Google.
it's not a bug
just a suggestion
I want to control the rocket to leave the current dimension when it reaches the specified height through configuration options, [because CubicChunks are installed, I set the height to 4096, but the rocket will leave the current dimension at 1200, so I want to request to modify the rocket transmission height distance ]:)

While GC itself sets this for each planet and moon it adds, as well as the overworld. All addons set and control that property for their own planets/moons/etc. And it could have a particular value set for a reason. Are you asking for this feature for the Overworld only or all planets/moons?

What I want is to be able to control this function in the configuration files of all dimensions, which can be used as the altitude required to leave the atmosphere of different planets:)

Since that would require addon devs to also make that a configurable option, this is something I don't see as being done in this version of Galacticraft.

Then I may need to build and modify it myself, sorry for your trouble


Print inputs to chain id

Print the inputs to the chain id, in order to isolate differences when chain ids differ. Command is advanced chain-id-inputs.
Sample output from a local testnet:
Genesis state hash: 3NKbZ5CdQducPnM5kzWF6C8fyK6zGtuo9Ez1yrYRq3iqmG5uf14H
Genesis_constants:
  Protocol:          {"k":290,"slots_per_epoch":7140,"slots_per_sub_window":7,"delta":0,"genesis_state_timestamp":"2021-03-16 00:45:51.072000Z"}
  Txn pool max size: 3000
  Num accounts:      250
Snark keys:
  d0f8e5c3889f0f84acac613f5c1c29b1
  922bd415f24f0958d610607fc40ef227
  06d85d220ad13e03d51ef357d2c9d536

Closes #8254.

!approved-for-mainnet


[Feature Request] Being able to upload file into client without having to send it to some server

I want to perform some actions on a file within the client and not send it anywhere. Would be cool if dropzone supported that. So far Dropzone requires an URL to post the data to.

This was one of the first things I wanted to do as well. It's not supported by default by Dropzone, so this Vue implementation also won't have it.
You can sort of work around this by setting auto-process-queue to false. Then create your own function to upload the files that are within the Dropzone. You can get a reference to them with this.$refs.yourdropzoneid.getAcceptedFiles(). This will not update the actual dropzone itself though, so no progressbars or anything.

Thanks for the suggested workaround @Ragura , that's the best I can think of at the moment as well.
There is a better way to do it as you can see that the dropzone homepage actually does it on their demo. I've had a look at their source code but it's not exactly clear as to how they're making it work. I'll keep thinking though

@Ragura , @rowanwins
Thank you two! For now I'm going to go with @Ragura  's workaround.


Render page method doesn't forward top and left coordinates

https://github.com/jbaarssen/PdfLibCore/blob/981ed44c1024de792da6ab0d78b5e212fa8e5113/src/PdfLibCore/PdfPage.cs#L79
should be
Pdfium.FPDF_RenderPageBitmap(renderTarget.Handle, Handle, rectDest.left, rectDest.top, rectDest.width, rectDest.height, orientation, flags);

I'll pick this up in the next release.

I've created a new release (2.50 which forwards top and left.


ChoiseParser in docs

It says here about the ChoiceParser, but I didn't find it in the package.
Was one of these parsers meant?

Good catch, yes, it seems I never updated the docs with this change. You'll want to use ExactChoiceParser() or PredictiveChoiceParser() instead of ChoiceParser().
I'm preparing to update this library to 1.9.21, so I'll get the docs updated with that release.

Fixed in bfc9143f5084e5de46e3633624a83976e544edaa


Fixup examples by making ir_init initialize the ret_type as well

ret_type always needs to be initialized to -1 for all return builders to work, not sure why it is done separately (immediately after ir_init) in php-src, this moves the init inside of ir_init, thus fixing the examples.

Examples are fixed in a bit different way via 7835df5831e24cd57628570f11d6c0d4a9593f2f. Thanks.


openssl

yl@yl-virtual-machine:~/sensor$ openssl version -a
openssl: /lib/x86_64-linux-gnu/libssl.so.3: version OPENSSL_3.2.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version OPENSSL_3.0.9' not found (required by openssl)
openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version OPENSSL_3.3.0' not found (required by openssl) openssl: /lib/x86_64-linux-gnu/libcrypto.so.3: version OPENSSL_3.2.0' not found (required by openssl)
### Tasks

how to fix it

This seems to be more a question about how to get OpenSSL installed rather than an issue with OpenSSL itself, and as such should be opened as a discussion rather than an issue.
In order for us to help you, we really need a bit more than some error messages: what system are you on, and what exactly have you done to get to this point?

Thanks for your reply. I added export OPENSSL=/usr/local/openssl/bin export PATH=$OPENSSL:$PATH:$HOME/bin to /etc/profile and sourced /etc/profile. However, when checking the version as shown in the title, it prompted that the relevant library could not be found. The above is the process of my problem (before this, I had an old version of openssl in my ubuntu, but I did not uninstall it but directly downloaded the new openssl 3.3.2). Then I added "export LD_LIBRARY_PATH=/usr/local/openssl/lib64" souce ~/.bashrc to ~/.bashrc to solve this problem.


Feat/pg core/query builder/update

:tada: This PR is included in version 3.22.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


IE & Edge - Shadow Blocks drawed out from toolbox are offset.

Expected Behavior
They Stay Aligned with parent block.
Actual Behavior
They are offset.
Steps to Reproduce
Launch Ede and draw a block from the toolbox.
Operating System and Browser
Windows 10 Edge

Fixed upstream


Console error when unmounting a SortableContext

Hi guys!
Some weeks ago we discuss problem "Console error when unmounting a DndContext" #637 and solution will be found. Nevertheless i got the same problem in SortableContext.
 This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in a useEffect cleanup function.

+1

Look at video to reproduce
If destroy tree component before back animation is completed you will get console error.

The problem is gone.
My current version:
    "@dnd-kit/core": "^6.0.3",
    "@dnd-kit/sortable": "^7.0.0",
    "@dnd-kit/utilities": "^3.2.0",

Thanks!


implement DB options for customizing maxopenfiles

Closes #50
Allow the DB client (i.e. Cosmos SDK) using DBOptions to setup  maxopenfiles to leveldb/rocksdb/pebble

@JayT106 thanks for the PR! want to let you know that cosmos-db within the sdk may not land till after twilight, does that work for you?

@marbar3778
yap, no worries.


Set aws file secrets as individual files

what
creates config and credentials as files within .aws instead of mounting entire secret as .aws.
why
This allows for other directories (.aws/cli, .aws/sso) which need to be created/used by the CLI itself for caching to exist.
tests
tested via various configuration of helm values to receive desired result.   Running in own environment the case where aws.config is specified.  Does assume that config/credentials is never specified along with awsSecretName.
references
closes #380

Thanks @ryan-dyer-sp! Can you please provide an example on values.yaml and also bump the chart minor version, I would consider this a feature.

Thanks @ryan-dyer-sp! Can you please provide an example on values.yaml
IDK what you're asking for here.  https://github.com/runatlantis/helm-charts/blob/main/charts/atlantis/values.yaml#L128-L139 isnt changing.  Its just mapping the individual elements within the secret to filenames instead of mounting the entire secret as a volume.
One behavior that does change with this is that secrets mounted with subPaths do not auto update (changes to the secret do not get updated within the container).  So you do have to restart the atlantis pod in this case.  Perhaps I should add the standard checksum annotation/label in order to force a redeploy when the config is updated?
Will update chart.yaml.

Thanks

Hi!

IDK what you're asking for here. https://github.com/runatlantis/helm-charts/blob/main/charts/atlantis/values.yaml#L128-L139 isnt changing. Its just mapping the individual elements within the secret to filenames instead of mounting the entire secret as a volume.

Please check my comment, I'm trying to understand if the values are supposed to change.

One behavior that does change with this is that secrets mounted with subPaths do not auto update (changes to the secret do not get updated within the container). So you do have to restart the atlantis pod in this case. Perhaps I should add the standard checksum annotation/label in order to force a redeploy when the config is updated? Will update chart.yaml.

This is a good idea 👍

@GMartinez-Sisti Anything else needed?

@GMartinez-Sisti Anything else needed?

I'll look into this soon! 😃

/remove stale
Chart version bumped

/remove stale

@GMartinez-Sisti re updated.  Anything else?

@GMartinez-Sisti re updated. Anything else?

Thanks @ryan-dyer-sp! What do you think about adding a unit test to ensure it works as expected and no one breaks it by mistake?


[URGENT] Update to v1.12

Necessary to support ACMEv2, as ACMEv1 is now deprecated and cannot be used

Thanks I will do it via a rebase to keep clean commit history with upstream.

Thanks for pinging me about this, new image coming.


Multiprocessing

Use multiprocessing to calculate on multiple cores at the same time to decrease runtime.
Multiprocessing code:
from multiprocessing import Pool
with Pool() as mp_pool:
    results = mp_pool.map(get_most_popular_color, image_paths)

Get number of CPUs (let's denote this number as "n")
Split primary modelpoint into n possibly equal parts.
We should create n instances of the model and calculate it on each chunk.
The results from the model should be in form of data frames. The model shouldn't yet save the results to csv.
Merge the n results into one.
Save to csv.

I think we shouldn't split modelpoint into n parts. We should run model with argument and model should take the subset of rows.

Working on this in feature/multiprocessing branch.
Todo:

merge multiple results into one result and save to csv files
add a setting whether or not to use multiprocessing
update run.py template
deal with status updates/logging


Fail Tealium helper silently

The approach in the existing module to throw the entire app is fairly aggressive. It means any error with your tealium config will ungracefully break the entire website when the Tealium tracking probably would never be so mission-critical to justify a break that substantial.
This is just a minor suggestion + patch. The code's changed to handle that error differently. If the config object is partially configured, we should log a warning and if there's no configuration we shouldn't run the integration at all.
An alternative would be to always warn instead of the throw, but to keep the console clean – this seems nicest in my opinion!

Was just wondering if this PR was dead, or if we could get a merge?


Add HTTP client

Description
Closes #14
Type of change

[x] New feature (non-breaking change which adds functionality)

Checklist

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my code
[ ] I have commented on my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

Pull Request Test Coverage Report for Build aa65527f4f719ff2d3f31719490665f1e14f66a5-PR-29

21 of 21 (100.0%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build 864e146ec846274d8900173756d23bdac2366c32:
0%

Covered Lines:
124

Relevant Lines:
124

💛  - Coveralls


Remove constraintlayout-compose for Kotlin Multiplatform Compatibility

Issue

close #270

Overview (Required)

Remove dependencies from all feature modules. 👋
Use additional Spacer to align the layout without ConstraintLayout.

Links

None

Screenshot

Before
After

You are the first contributor after being public!!🏆


Can not logon to sqrljava.com

I was not able to logon to
https://sqrljava.com:20000/sqrlexample/login
The sqrl:// header seems to be misunderstood. In Firefox (65.0) on Ubuntu 18.04 I get an can not load page error. With Chrome I get the ´waiting for confirmation´ page, but it tries to open ´xdg-open´ . wen I just power through I get the following error

https://sqrl-test.herokuapp.com/    works fine ;)

What version of the webextension were you using?
Any special things i need to do to reproduce this? (havent been able to reproduce yet on macOS)

Ah i can reproduce the problem.
On Firefox/Ubuntu, after clicking the "Sign in with SQRL" image, the page is redirected to a sqrl:// page like you said.
I shall investigate.

in chrome on ubuntu even more stuff happens. In Firefox I have 0.0.35, and in chrome i seem to have 0.0.32 as stated on the help page, but it shows as 0.0.34 in the extension list.
I updated the sourcefolder and got some files, I imported those and and now it states 0.0.34 in the help.
When I try https://sqrljava.com:20000/sqrlexample/login I still get xdg-open, when i cancel that the logon works.

I've contacted the owner of sqrljava.com to see if we can work on this.
He cannot reproduce the problem at the moment.
I can reproduce on Firefox for Ubuntu and Mac.


Tag pages miss articles due to case-sensitiveness

middleman-blog's BlogData is grouping articles for each tag in a case-sensitive way - ie, foo and Foo are two different tags.
But the destination path of the page is case-insensitive - ie, both foo and Foo go to /blog/with-tag/foo.html.
The result is that you get only one page for those tags (say, just foo), but missing the other tag's articles (ie, you don't get the Foo articles shown).
I'd expect tags to be case-insensitive - so I'd add .downcase in both related snippets. You get a single foo tag (that could be shown as Foo or not), that shows both foo and Foo articles. About wether to show foo or Foo as the tag name, we can use the first/last tag's format - I don't care. If the casing shown isn't the one the user wants, they can always re-tag the articles so it matches.

Hiya @matiasgarciaisaia
I see what you mean I have replicated this, and I think it makes sense in terms of what you are saying - the other option is to create Tag files that are not case sensitive, but thats a poor option for URL continuity.
Could you please remove the PRs you have done and resubmit a new PR along with a Feature test for this submission.

Ian, I'm actually not keen at all doing feature tests.
I can totally rework the PR, but I'd need your assistance to create the
feature tests. Do you have any test that's too similar to the ones I should
create? Or would it be OK if I give collaborators access to edit my PR so
you can do the tests for me?
Thank you very much

I don't think there are any Fixtures for this - but basically we want to test that Posts tagged with essentially the same word 'Foo' | "foo" bring up the same list of posts on the Tag template. And that these Tags link to the correct template.
Probably add to this set of tests
https://github.com/middleman/middleman-blog/blob/master/features/tags.feature
Whats the link to your Blog / Repo please?


AWS SES (smtp) not working

Hi,
We have configured the AWS SES (Smtp) for email notification it's not working, tried with port 587, 465, 25 but no luck.
Any input here.

Did you try with "Secure" unchecked?

yes, but no luck

any error msg?

I can't see any error in AWS ECS we have hosted the tool in AWS Farget.  only monitoring log on ping and URL reporting to CloudWatch log groups.

After clicked the "Test" button, there should be a error msg in bottom right.

Hi louislam,
Here is the test screenshot for your reference.

Just for your update, we have tested slack integration it's working fine.

So there is no error msg in the bottom right corner?  The red box.

No error

Um, should be something show up whether is success or not.
After clicked "Test" button, did you see "Sent Successfully"?
If nothing show up, try smtppro.zoho.com 465 with random username/password

Thank louislam,
I can see the connection time out error in SMTP configuration, it looks like the issues with the network level.

We have tried Gmail but still, it is blocked somewhere,
The same tool I have set up inside the windows desktop docker is running fine, tested the AWS SES, and Gmail working fine.  Will check this with our network team.


AttributeError: '_SpikeTensor' object has no attribute 'is_mps'

snntorch version: 0.6.1
Python version: 3.7
Operating System: linux

Description
When I tried to run snntorch/examples/tutorial_7_neuromorphic_datasets.ipynb, I got an error training the neural network
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/tmp/ipykernel_20075/2455384482.py in <module>
     13
     14         net.train()
---> 15         spk_rec = forward_pass(net, data)
     16         loss_val = loss_fn(spk_rec, targets)
     17

/tmp/ipykernel_20075/3960929974.py in forward_pass(net, data)
      6
      7   for step in range(data.size(0)):  # data.size(0) = number of time steps
----> 8       spk_out, mem_out = net(data[step])
      9       spk_rec.append(spk_out)
     10

~/anaconda3/envs/d/lib/python3.7/site-packages/torch/nn/modules/module.py in _call_impl(self, *input, **kwargs)
   1100         if not (self._backward_hooks or self._forward_hooks or self._forward_pre_hooks or _global_backward_hooks
   1101                 or _global_forward_hooks or _global_forward_pre_hooks):
-> 1102             return forward_call(*input, **kwargs)
   1103         # Do not call functions when jit is used
   1104         full_backward_hooks, non_full_backward_hooks = [], []

~/anaconda3/envs/d/lib/python3.7/site-packages/torch/nn/modules/container.py in forward(self, input)
    139     def forward(self, input):
...
--> 462         elif arg.is_mps:
    463             arg = arg.to("cpu")
    464         arg = torch.Tensor(arg)  # wash away the SpikeTensor class

AttributeError: '_SpikeTensor' object has no attribute 'is_mps'

How can I solve this problem?

For an immediate fix, update your version of PyTorch to PyTorch>=1.13.
I'll try to have an alternative fix that enables compatibility with older versions shortly.

For an immediate fix, update your version of PyTorch to PyTorch>=1.13.
I'll try to have an alternative fix that enables compatibility with older versions shortly.

Thank you very much for your quick reply.

Fixed in snntorch==0.6.2 onwards.
You do not need to update PyTorch if you update snnTorch.
See #200 for more details.


Fix Landing page missing space character

Action Items
Add space between "onGithub." on the landing page.
Resources/Instructions
As currently seen at http://civictechindex.org/ the on and Github are stuck together:

Hi ! I'm starting to contribut to open source projects and I found this issue.


Alternate Accounts

I have visual studio subscription with my office account and i have added my personal Microsoft account as an alternate account, will i be able to view my office VS Azure Subscription when i login to azure using the Microsoft account ?

@muralidaran -- it looks like you're having an issue with the product itself, rather than an issue with the documentation. Here are a few support options that might help you with the issue that you're experiencing:

If you have a subscription, contact Visual Studio Subscriptions Support.
If you are not sure if you have a subscription, sign in to https://my.visualstudio.com/subscriptions to see all the subscriptions assigned to your email address.
If you do not have a subscription, consider using either our Live Chat (English only) support option for installation issues, or reaching out to us and other Visual Studio developers through our Visual Studio conversation in the Gitter community.


CMake build types are hard-coded case-sensitive

Hello, I've got a build setup where the build type is being passed as lower case for $reasons. CMake <= 3.21 explicitly says CMAKE_BUILD_TYPE is case insensitive (3.22 and greater say "Depending on the situation, the value of this variable may be treated case-sensitively or case-insensitively" which is about as helpful as you'd expect).
In OCIO's CMakeLists.txt the following check essentially forces it to be case sensitive, which breaks my build:
if(NOT "${CMAKE_BUILD_TYPE}" IN_LIST CMAKE_CONFIGURATION_TYPES)
    string(REPLACE ";" ", " _CMAKE_CONFIGURATION_TYPES_STR "${CMAKE_CONFIGURATION_TYPES}")
    message(FATAL_ERROR
            "CMAKE_BUILD_TYPE=${CMAKE_BUILD_TYPE} is unsupported. Supported values are: ${_CMAKE_CONFIGURATION_TYPES_STR}.")
endif()

which is especially annoying because you do an explicitly case-insensitive check for "Debug" directly underneath :)
set(_BUILD_TYPE_DEBUG OFF)
if(CMAKE_BUILD_TYPE MATCHES "[Dd][Ee][Bb][Uu][Gg]")
    set(_BUILD_TYPE_DEBUG ON)
endif()

Is that check on the build type really necessary? It's not an idiom I've seen elsewhere. I started trying to figure out how to rewrite it cleanly in a case-insensitive way but cmake gave me a headache so I just removed it completely on my fork to get my build working.

I would also vote to remove that check as you proposed, I'm not sure it should be the project's responsibility to check that. @hodoulp  do you have reasons to keep that check (seeing that you included it in a previous PR)?

The goal of the check is to validate that the build is doing what it's expected to do. If a dev requests debug then the build must do in debug mode or fail. If a dev request an unknown build type (e.g. debugg) then it must fail right away and not fallback to a default build type.
Ideally, the check must be case insensitive.

This is now fixed via PR #1765.  Thanks for reporting @anderslanglands !


feat: Support shuffling of dns results order to avoid traffic imbalance

Also related to https://github.com/seanmonstar/reqwest/issues/1007

Please assign to me ^_^

In the trust-dns library, AsyncResolver is commonly used for DNS resolution in asynchronous environments. There is a property called ServerOrderingStrategy in ResolverOpts configuration, whose default value is QueryStatistics.
pub enum ServerOrderingStrategy {
    /// Servers are ordered based on collected query statistics. The ordering
    /// may vary over time.
    QueryStatistics,
    /// The order provided to the resolver is used. The ordering does not vary
    /// over time.
    UserProvidedOrder,
}

When using trust-dns for DNS resolution in OpenDAL, DNS configuration is parsed from the system conf file.
#[cfg(feature = "trust-dns")]
impl AsyncTrustDnsResolver {
    pub fn new() -> io::Result<Self> {
        let resolver = trust_dns_resolver::TokioAsyncResolver::from_system_conf(
            trust_dns_resolver::TokioHandle,
        )?;

        Ok(Self {
            inner: Arc::new(resolver),
        })
    }
}

During this initialization process, the ServerOrderingStrategy can be manually set to change the order of DNS servers.
For UserProvidedOrder, in fact, no operation is performed to change the order, and the order of DNS servers remains unchanged.
Therefore, if we want to implement DNS shuffle, we may need to modify the implementation of the UserProvidedOrder in trust-dns so that it can use the user-defined order to change the order of servers.


Linux, python 3.11

Getting this error in a loop, every few minutes:
Tue Nov 14 11:56:29 2023  GET request received from: ('<IP_ADDRESS>', 45378)
Tue Nov 14 11:56:29 2023  Sending GET to xxxxx
----------------------------------------
Exception occurred during processing of request from ('<IP_ADDRESS>', 45378)
Traceback (most recent call last):
  File "/opt/lib/python3.11/socketserver.py", line 317, in _handle_request_noblock
  File "/opt/lib/python3.11/socketserver.py", line 348, in process_request
  File "/opt/lib/python3.11/socketserver.py", line 361, in finish_request
  File "/opt/lib/python3.11/http/server.py", line 671, in __init__
  File "/opt/lib/python3.11/socketserver.py", line 755, in __init__
  File "/opt/lib/python3.11/http/server.py", line 436, in handle
  File "/opt/lib/python3.11/http/server.py", line 424, in handle_one_request
  File "/opt/etc/edgebridge/edgebridge.py", line 514, in do_GET
    proc_msg(self)
  File "/opt/etc/edgebridge/edgebridge.py", line 490, in proc_msg
    handle_requests(server)
  File "/opt/etc/edgebridge/edgebridge.py", line 421, in handle_requests
    proc_forward(server, method, path, arglist[0])
  File "/opt/etc/edgebridge/edgebridge.py", line 170, in proc_forward
    headers = build_headers(server, path)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/etc/edgebridge/edgebridge.py", line 148, in build_headers
    headers['Host'] = path.split('//')[1].split('/')[0]
                      ~~~~~~~~~~~~~~~~^^^
IndexError: list index out of range

<IP_ADDRESS> - IP of the SmartthIngs hub.

Ah, sorry, just checked, it is from the Edge Weather V1. Haven`t configured it there fully. yet.
Closing


Using craco with create-react-app in production on heroku fails

Context:

I'm newer to Craco, so apologies if any of these are dumb questions!
I've been using TailwindCSS and followed their instructions precisely on setup, located here https://tailwindcss.com/docs/guides/create-react-app
I'm trying to deploy a simple static website powered by React via Heroku using Craco+Tailwind

Issue: Both dev and production builds work locally, but the moment I deploy to heroku, it fails. I get this error when visiting the app:

And my logs look like this:

And my craco config (per Tailwind docs) looks like this:
module.exports = {   style: {     postcss: {       plugins: [require("tailwindcss"), require("autoprefixer")],     },   }, };
Potential cause: It looks like Heroku is launching a dev build in production. Some notes:

Heroku logs indicate "starting a development server"
the error on the app suggests the React Refresh exists when it shouldn't in a production environment
switching the scripts from craco back to react-scripts resolves production environment issues, and app works again
In this thread, one of the comments is about Craco hiding some build scripts but tbh I have no idea if relevant here https://github.com/DocSpring/craco-less/issues/10

Steps to repro

create a brand new react project using npx create-react-app test
add tailwind following the instructions precisely
deploy all the boilerplate to heroku

Expected behavior: just works as it does locally
Actual behavior: builds succeed, but app crashes due to above errors
Question: Is this something I need to fix via craco's config file? I can't seem to figure it out with the documentation and would really appreciate some help.

https://medium.com/jeremy-gottfrieds-tech-blog/tutorial-how-to-deploy-a-production-react-app-to-heroku-c4831dfcfa08
NVM, basically fixed it via this.  This was more a heroku problem of serving the dev app instead of build.

https://medium.com/jeremy-gottfrieds-tech-blog/tutorial-how-to-deploy-a-production-react-app-to-heroku-c4831dfcfa08
NVM, basically fixed it via this.  This was more a heroku problem of serving the dev app instead of build.

I'm trying to deploy a react app with heroku and tailwind as well - getting issues with craco I believe. How did you get it figured out? https://github.com/corgimaman/Scissors-Palace/ Mine has a front and backend.


feat: add indicator r018

closes #90

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

17 of 17 (100.0%)  changed or added relevant lines in 2 files are covered.
45 unchanged lines in 3 files lost coverage.
Overall coverage increased (+0.05%) to 90.456%

Files with Coverage Reduction
New Missed Lines
%

src/indicators/r028.rs
1
95.45%

src/indicators/mod.rs
7
78.57%

src/lib.rs
37
91.52%

Totals

Change from base Build<PHONE_NUMBER>:
0.05%

Covered Lines:
1289

Relevant Lines:
1425

💛  - Coveralls


always get acc,acc_norm, perplexity =1 on triviaqa task based on llama2 model

I use the following command to run triviaqa task
lm_eval --model hf \     --model_args pretrained=../llama/models_hf/7B \     --tasks triviaqa \     --num_fewshot 1 \     --device cuda:2 \     --batch_size 8
I just get acc_norm=1, it's same when I use acc or perplexity indicator.

Hi, Thank you for your response. with the exact_match indicator, I get value=0.07.

here is the YAML I used, I only changed the metric.
task: triviaqa
dataset_path: trivia_qa
dataset_name: rc.nocontext
output_type: generate_until
training_split: train
validation_split: validation
doc_to_text: "Question: {{question}}?\nAnswer:"
doc_to_target: "{{answer.aliases}}"
should_decontaminate: true
doc_to_decontamination_query: question
generation_kwargs:
  until:
    - "\n"
    - "."
    - ","
  do_sample: false
  temperature: 0.0
filter_list:
  - name: remove_whitespace
    filter:
      - function: remove_whitespace
      - function: take_first
target_delimiter: " "
metric_list:
  - metric: acc
    aggregation: mean
    higher_is_better: true
    ignore_case: true
    ignore_punctuation: true
metadata:
  version: 2.0

For the code, I used the following command and changed nothing except the above YAML file.
git clone https://github.com/EleutherAI/lm-evaluation-harness.git

those metrics you used are currently only supported for loglikelihood or multiple_choice output_type tasks. In your case, this could be typically achieved by setting output_type: loglikelihood, but is complicated by triviaqa’s use of multiple gold standard answers.
We’ll make sure that running this will error out explicitly in future to avoid confusion.
We are also working on making metrics more easy to understand and add in the library in general!

Thanks a lot.


Fix for issue 1775: in ldapquery, validation out of bounds query should be case insensitive as ldap is.

Any compare operation in ldap should be case insensitive.
This is to fix a well known issue in oc groups sync command that is comparing base dn to a dn and showing out of bounds if the compare operation fails.

/assign

I will do. Thanks for reviewing

/lgtm
Passing to @deads2k for final review


Connect to Chain from given point (in the past)

What & Why
Currently, it is mandatory for all heads to be up-and-running to observe any of the on-chain activity. This requires extra coordination between participants and may lead to situations where it's easy to miss an init for example.
To make the experience more delightful and reliable, we want to introduce the ability to start a hydra-node, not from the most recent chain tip, but from a given tip in the past. Allowing thereby the node to replay on-chain events it could have missed.
Tasks

#299
#302

TODO: Update the changelog.


Gaussian splats not pickable

In gsplat-compressed-material.js there is the following code:
        #ifdef PICK_PASS
            if (B < 0.3) {
                discard;
            }
            return uColor;
        #endif

Where its expecting the colour picker color as uColor however this is no longer how the picker works and instead the color is derived from the mesh_instance_id parameter from
getPickOutput()

Steps to Reproduce

add a gaussian splat
attempt to pick it
the pick colour is rendered black and no selection picked

Interesting theres no equivalent in gsplat-material.js.

yep definitely something we need to fix.


Document how we communicate

When should we use email or Hipchat or Jira or Github or Hangouts or something else?

@lewisnyman, Fredrik and I have put some content together in the past. It might be of use here: https://docs.google.com/a/flink-solutions.de/document/d/1r5nua_AZcTNeUbUV_uBZB0NrzPZHz_l9Qg1O9oMijn8/edit?usp=sharing and https://docs.google.com/a/wunderkraut.com/document/d/1hXt1FIfPJNfLMUzTYH7NujNr0a8QlEi0tQjxidcLHy4/edit?usp=sharing

This looks great! Maybe you could start with a pull request to move this into the Wunder Way? Happy to review.

Sure. I have started a new branch in my repo and will create a PR upon the finalization of the first version.

I think this is closed by #107 - true?

@joesb, yes. Should I have reference 103 in the PR?

Great, good news.
GitHub does have a special syntax for closing issues from Pull Requests (I keep forgetting to use it myself, too!):
https://github.com/blog/1506-closing-issues-via-pull-requests
https://help.github.com/articles/closing-issues-via-commit-messages/

Thanks
On Wed, Aug 19, 2015 at 12:40 PM Joe Baker<EMAIL_ADDRESS>wrote:

Great, good news.
GitHub does have a special syntax for closing issues from Pull Requests (I
keep forgetting to use it myself, too!):
https://github.com/blog/1506-closing-issues-via-pull-requests
https://help.github.com/articles/closing-issues-via-commit-messages/
—
Reply to this email directly or view it on GitHub
https://github.com/wunderkraut/WunderWay/issues/103#issuecomment-132535335
.

--

Florian Huber - Project Manager, Wunderkraut Germany
<EMAIL_ADDRESS>// phone +49.179.4698130
Wunderkraut. Working wonders with Drupal – http://wunderkraut.com/
Wunderkraut GmbH, MTZ D1.019, Agnes-Pockels-Bogen 1, D-80992 München
HRB191094 Amtsgericht München, Geschäftsführer: Reinhard Gloggengiesser,
Daniel Nolde


Underwhelmed by the misuse of versionType and version to support purl

Like many, I was very excited to explore the 5.1 release with purported support for package url. But it turned out to be just a couple of string attributes versionType and version, that can be used to populate with any values without any validations. In fact, versionType could be purl, package url, PURL, anything. While purl specification has no limit on the length, version attribute has a max length of 1024, which would limit the number of qualifiers (Example repository_url=full url) that can be used.
I think if we are serious about replacing CPE with purl, it deserves a first party attribute with correct validation rules. I would appreciate if you revisit the purl support for 5.2 release.

Is it possible to check if something is a valid PURL using JSON-schema rules?

Thank you for the prompt response. If the attribute is called purl that alone is usually sufficient for the downstream tools to use appropriate validation.
Below is how CycloneDX handles the various identifiers.
https://github.com/CycloneDX/specification/blob/master/schema/bom-1.6.schema.json#L972-L999
CVE spec could support an array of purls, omnibor, and swhid etc similar to cpes.

Is it possible to check if something is a valid PURL using JSON-schema rules?

Fwiw, the CSAF spec has a basic regex validator for purls:

https://docs.oasis-open.org/csaf/csaf/v2.0/os/csaf-v2.0-os.html#31334-full-product-name-type---product-identification-helper---purl
https://github.com/oasis-tcs/csaf/blob/f7c818282c13ac68f14a732708d25b4359f74b02/csaf_2.0/json_schema/csaf_json_schema.json#L249-L256


Is it a Bug to compute the distillation loss?

I feel confused about compute_distillation_crossentropy_loss.
Let me call the main model as target_model and the model to compute the distillation loss as sub_model.
From the code, it seems the target_model compute logits with rois (https://github.com/kshmelkov/incremental_detectors/blob/master/frcnn.py#L143).
The sub_model computes distillated_logits with proposals
(https://github.com/kshmelkov/incremental_detectors/blob/master/network.py#L104)
The rois and proposals are with same shape but different contents. However, the distillation loss is computed from them in compute_distillation_crossentropy_loss().
I am not quite sure about this point. I would be grateful it if anyone can give me some hint or discussion.

Sorry It was my bad. I figure it out you merge the rois and proposals in https://github.com/kshmelkov/incremental_detectors/blob/master/network.py#L127


Stack Overflow caused by repeated calls to default Java serialized data reader

When we run PageRank for too many iterations (100 iterations consistently triggers it), we get a stack overflow that stems from reading deeply nested serial data using the default java serializer. Basically, we get repeated calls to the following sequence of methods
        at scala.collection.immutable.$colon$colon.readObject(List.scala:435)
        at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1015)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
        at scala.collection.immutable.$colon$colon.readObject(List.scala:435)
        at sun.reflect.GeneratedMethodAccessor1.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1015)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
        at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)

@ankurdave poked around a little bit and he said that Kryo seemed to be getting called on most things but apparently not whatever is causing this.
I did some more poking around today and when profiling PageRank running on livejournal, I found that a bunch of exceptions were being thrown in the Kryo serialization code. It seems that Kryo keeps trying to serialize something, fails with a NoSuchMethodException, and then falls back to Java serialization code. I'm thinking that these two might be related? The exception stack trace is here.

I have this exact same issue while running MLLib's ALS for many iterations.


Review old examples

Some examples which do not appear in the online documentation may be outdated. Review and update them:

dipatcher
ventilator

@RadW2020 Those examples are now up-do-date. Although they are out of the documentation for now, until I decide whether architectural examples/patterns should be included in osBrain documentation or should be set apart in a new repository containing commonly used architectures or communication patterns.


Overcome issue of refresh tokens expiring

The test suite has to expose credentials in one form or another and the preferred option of using refresh tokens currently suffers from the fact that they expire after 3 days.
Could we set up a Mock IdP that allows the test suite to log in and get access tokens without user intervention? It should only support the 2 users (alice & bob) needed for testing.

Could we set up a Mock IdP

It might be easy to do that with CSS: it's been designed to create custom configurations, such as indeed one where there are two mock users.
The drawback would then be that CSS would test its own tokens. (However, the software generating them versus the software validating them happen to be different stacks.)

Another option might be to fake system time though, such that we can use fixed tokens. Viable?

That might be OK in a container but I don't think it would be so good for someone working on tests locally. Currently I have to bootstrap every 3 days to get new refresh tokens and I'd like to avoid that for test developers.

The tests for ESS, CSS, NSS now all use inrupt.net as the IdP instead of using refresh tokens. We may look again at this but this change moves us closer to being able to run in a CI environment where we can pass in the username/password as secrets to that process.


Add CI workflows

@dhia-gharsallaoui maybe we can use the mage file in CI flow? LGTM otherwise!

@dhia-gharsallaoui fixed magefile for build https://github.com/grafadruid/go-druid/pull/74


[CWS] re-enable ebpfless-no-seccomp tests

What does this PR do?
This PR re-enable ebpfless-no-seccomp tests no that they should be less flaky (or if they still are with the goal of collecting data to understand why). A debug log is also changed into an error log so that we can see it in the logs if we ever trigger it.

Motivation

Additional Notes

Possible Drawbacks / Trade-offs

Describe how to test/QA your changes

/merge


[3.x] Backport some APIs in math structs

Backport get_center to AABB and Rect2 in core and C# (originally added in #52878).
Backport limit_length to Vector2 and Vector3 in core and C# (originally added in #45624).

Document and warn about Vector2's clamped being deprecated since limit_length is a rename of it.

Backport Color >= and <= operators in C# only (> and < are implemented for C# in 3.x but not in core).

Thanks!


Adding token based authentication

Some features like creating tests should only be restricted to the teacher who is logged in and has their account on Manthan.
I would like to use token-based authentication with a JWT token. If anyone has any other idea of Role-based authentication please suggest, otherwise I would like to work on the token-based authentication system under GSSOC.

@chauhanprakhar Some of the contributors are working on this issue under #83, you can interact with them if you like.