feat(api): unify Bedrock provider using Runtime API

Problem:
The current Bedrock implementation uses the Bedrock SDK, which requires separate handling for different model types and doesn't provide a unified streaming interface.
Solution:
Integrate the Bedrock Runtime API to provide a single, unified interface for all Bedrock models (Claude and Nova) using the ConverseStream API. This eliminates the need for separate handlers while maintaining all existing functionality.
Key Changes:

Refactored AwsBedrockHandler to use @aws-sdk/client-bedrock-runtime
Enhanced bedrock-converse-format.ts to handle all content types and properly transform between Anthropic and Bedrock formats
Maintained cross-region inference support with proper region prefixing
Added support for prompt caching configuration
Improved AWS credentials handling to better support default providers
Added proper error handling and token tracking for all response types

Dependencies:

Added @aws-sdk/client-bedrock-runtime for unified API access
Removed @anthropic-ai/bedrock-sdk dependency

Testing:

Verified message format conversion for all content types
Tested cross-region inference functionality
Validated streaming responses for both Claude and Nova models

This change simplifies the codebase by providing a single, consistent interface for all Bedrock models while maintaining full compatibility with existing features.

[!IMPORTANT]
Refactor Bedrock provider to use Bedrock Runtime API for unified model handling, improved error handling, and enhanced AWS credentials management.

Behavior:

Refactored AwsBedrockHandler in bedrock.ts to use @aws-sdk/client-bedrock-runtime for unified model handling.
Added prompt caching configuration and improved AWS credentials handling.
Enhanced error handling and token tracking for all response types.

Transformations:

Added convertToBedrockConverseMessages and convertToAnthropicMessage in bedrock-converse-format.ts for message format conversion.

Models:

Updated bedrockModels in api.ts to include new model configurations and pricing.

Dependencies:

Added @aws-sdk/client-bedrock-runtime and removed @anthropic-ai/bedrock-sdk in package.json.

This description was created by  for 8280a80c93c12dd9776b8de5a8f4a40c43a05061. It will automatically update as commits are pushed.

@Premshay Would you add some unit tests for your changes please?  Specifically for changes in:
src/api/transform/bedrock-converse-format.ts
src/api/providers/bedrock.ts

@Premshay Would you add some unit tests for your changes please? Specifically for changes in: src/api/transform/bedrock-converse-format.ts src/api/providers/bedrock.ts

Added unit tests. bedrock-converse-format tests all pass. bedrock.ts most tests fail, but real-life testing shows everything works - messages, cline tools are working, images are analyzed properly. All real-life tests pass, for all claude and nova models. So I think that the tests aren't so good and should be refactored.
it would also be nice to a video analysis support (Nova support video, and the code already includes the basic use, now we need to get Cline to allow uploading videos like images).
Easy Future additions are adding more bedrock models, i.e. Llama 3.2, etc.
And after Amazon releases from preview the prompt caching an intelligent prompt routing features, we can add those quickly and allow cost savings for all.

@Premshay Would you add some unit tests for your changes please? Specifically for changes in: src/api/transform/bedrock-converse-format.ts src/api/providers/bedrock.ts

Added unit tests. bedrock-converse-format tests all pass. bedrock.ts most tests fail, but real-life testing shows everything works - messages, tool

@Premshay Would you add some unit tests for your changes please? Specifically for changes in: src/api/transform/bedrock-converse-format.ts src/api/providers/bedrock.ts

Sorry for the lay, just saw that th tests weren't committed. You will see them now, @mrubens, @ColemanRoo

@Premshay Would you add some unit tests for your changes please? Specifically for changes in: src/api/transform/bedrock-converse-format.ts src/api/providers/bedrock.ts

Sorry for the delay, just saw that the tests weren't committed. You will see them now, @mrubens, @ColemanRoo

Hey @Premshay thank you for adding the tests!  Can you please make sure to merge latest Roo-Cline/main into your branch and then increment the version number in package.json
Also make sure you are not committing any failing tests as your PR will not be allowed to merge with failing unit tests

@Premshay Would you add some unit tests for your changes please? Specifically for changes in: src/api/transform/bedrock-converse-format.ts src/api/providers/bedrock.ts

Sorry for the delay, just saw that the tests weren't committed. You will see them now, @mrubens, @ColemanRoo

Hey @Premshay thank you for adding the tests! Can you please make sure to merge latest Roo-Cline/main into your branch and then increment the version number in package.json Also make sure you are not committing any failing tests as your PR will not be allowed to merge with failing unit tests

Done. Also added Meta 3, 3.1, 3.2 models available on bedrock api.

@Premshay Would you add some unit tests for your changes please? Specifically for changes in: src/api/transform/bedrock-converse-format.ts src/api/providers/bedrock.ts

Sorry for the delay, just saw that the tests weren't committed. You will see them now, @mrubens, @ColemanRoo

Hey @Premshay thank you for adding the tests! Can you please make sure to merge latest Roo-Cline/main into your branch and then increment the version number in package.json Also make sure you are not committing any failing tests as your PR will not be allowed to merge with failing unit tests

Updated the tests and all are passing now.

Had to redo the commits because of compounding unresolvable conflicts. It should now be ok.

why is this PR not merged yet?
Nothing is missing, and versions are being updated everyday, making this PR backwards in versions package each time...
How can we merge it?

why is this PR not merged yet?
Nothing is missing, and versions are being updated everyday, making this PR backwards in versions package each time...
How can we merge it?

why is this PR not merged yet? Nothing is missing, and versions are being updated everyday, making this PR backwards in versions package each time... How can we merge it?

Hi @Premshay Sorry for the delay; I am finally getting around to testing your PR and I noticed that when I ran through a simple task of having Roo-Cline write tests for me, the tokens and API costs in the screenshot

were not updating as the task moved forward.
I am also a bit concerned with this PR as the numbers in the screenshot do not feel accurate to how long the task took.
It seems like those values only updated after the final call instead of incrementally.
Can you please take a look at why this might be happening?  As of right now, I do not feel confident in merging this PR.

why is this PR not merged yet? Nothing is missing, and versions are being updated everyday, making this PR backwards in versions package each time... How can we merge it?

Hi @Premshay Sorry for the delay; I am finally getting around to testing your PR and I noticed that when I ran through a simple task of having Roo-Cline write tests for me, the tokens and API costs in the screenshot  were not updating as the task moved forward. I am also a bit concerned with this PR as the numbers in the screenshot do not feel accurate to how long the task took. It seems like those values only updated after the final call instead of incrementally.
Can you please take a look at why this might be happening? As of right now, I do not feel confident in merging this PR.

What you are describing sounds like every time I have used openrouter or Bedrock before. In other words: This seems to be a Cline issue in general. That's the way Cline works on calculating the usage numbers it gets.
What in the numbers seems to not make sense to you?
I've tested the different models, and I can see the differences as expected.
In any case I'll go through it again and see if the usage info can be further improved.

why is this PR not merged yet? Nothing is missing, and versions are being updated everyday, making this PR backwards in versions package each time... How can we merge it?

Hi @Premshay Sorry for the delay; I am finally getting around to testing your PR and I noticed that when I ran through a simple task of having Roo-Cline write tests for me, the tokens and API costs in the screenshot  were not updating as the task moved forward. I am also a bit concerned with this PR as the numbers in the screenshot do not feel accurate to how long the task took. It seems like those values only updated after the final call instead of incrementally.
Can you please take a look at why this might be happening? As of right now, I do not feel confident in merging this PR.

What you are describing sounds like every time I have used openrouter or Bedrock before. In other words: This seems to be a Cline issue in general. That's the way Cline works on calculating the usage numbers it gets. What in the numbers seems to not make sense to you? I've tested the different models, and I can see the differences as expected. In any case I'll go through it again and see if the usage info can be further improved.

In regards to the numbers, for the task I had your PR version do, it wrote ~400 lines of brand new code; it iterated over the code multiple times and ran tests and coverage reports each time.  It was quick, but each time it accomplished a task, the Takens: and API Cost values should have updated (which they did not) and those numbers only updated at the end.  And from my previous experiences have Cline perform test writing tasks, the numbers that DID eventually appear at the end seem too low to me, indicating it did not aggregate each call to openrouter and only recorded the last one.

Thanks for the added information.
One point to be sure: you wrote "to openrouter". You meant "to bedrock", right?

Thanks for the added information. One point to be sure: you wrote "to openrouter". You meant "to bedrock", right?

The key I use is an openRouter API Key, so if that routes differently in your PR, then that could be an issue as well.  But my assumption is that bedrock is just the grouping of all the different models?
But either way, using the same key I should get the same experience between existing marketplace version and your PR version.

I'm sorry, I must be missing something here.
My PR is only for using the Bedrock service. It can only be used providing
aws credentials, and the only models that will work would be those that the
person with those credentials have access too. Otherwise you get specific
errors stating lack of credentials or non access to model. The Bedrock
provider can't be checked with anything but AWS credentials.
Could you further explain?
On Mon, Dec 16, 2024, 22:41 Mike C @.***> wrote:

Thanks for the added information. One point to be sure: you wrote "to
openrouter". You meant "to bedrock", right?
The key I use is an openRouter API Key, so if that routes differently in
your PR, then that could be an issue as well. But my assumption is that
bedrock is just the grouping of all the different models?
But either way, using the same key I should get the same experience
between existing marketplace version and your PR version.
—
Reply to this email directly, view it on GitHub
https://github.com/RooVetGit/Roo-Cline/pull/60#issuecomment-2546697657,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGWMILCZDELXJXE3PNLSSUT2F43G7AVCNFSM6AAAAABTLSYLP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBWGY4TONRVG4
.
You are receiving this because you were mentioned.Message ID:
@.***>

I'm sorry, I must be missing something here. My PR is only for using the Bedrock service. It can only be used providing aws credentials, and the only models that will work would be those that the person with those credentials have access too. Otherwise you get specific errors stating lack of credentials or non access to model. The Bedrock provider can't be checked with anything but AWS credentials. Could you further explain?
…
On Mon, Dec 16, 2024, 22:41 Mike C @.> wrote: Thanks for the added information. One point to be sure: you wrote "to openrouter". You meant "to bedrock", right? The key I use is an openRouter API Key, so if that routes differently in your PR, then that could be an issue as well. But my assumption is that bedrock is just the grouping of all the different models? But either way, using the same key I should get the same experience between existing marketplace version and your PR version. — Reply to this email directly, view it on GitHub <#60 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGWMILCZDELXJXE3PNLSSUT2F43G7AVCNFSM6AAAAABTLSYLP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBWGY4TONRVG4 . You are receiving this because you were mentioned.Message ID: @.>

hmmm, ok
Could you go into details exactly how one would go about using the bedrock service?  We need to be able to test these changes...
However, I am still concerned that my current experience differs between your PR and the currently available version when it comes to tracking tokens and cost as the task progresses.

I can't really explain why you're seeing differences. Where do you see the
difference? I mean, which provider are you choosing? Openrouter?
Personally I came across that same behavior using openrouter in Cline (the
original marketplace Cline), for the past 2 months.
As for why you see difference between versions, I can't say, my PR only
handles the changed the Bedrock provider file and added a new file for
bedrock transform. You can see in the files changed that no other provider
or src file are committed.
As for how to test: you need to choose bedrock provider, input credentials,
and choose one of the models from the list (currently includes amazon nova,
claude and Meta Llama models).
On Mon, Dec 16, 2024, 23:01 Mike C @.***> wrote:

I'm sorry, I must be missing something here. My PR is only for using the
Bedrock service. It can only be used providing aws credentials, and the
only models that will work would be those that the person with those
credentials have access too. Otherwise you get specific errors stating lack
of credentials or non access to model. The Bedrock provider can't be
checked with anything but AWS credentials. Could you further explain?
… <#m_6693138867674315585_>
On Mon, Dec 16, 2024, 22:41 Mike C @.> wrote: Thanks for the added
information. One point to be sure: you wrote "to openrouter". You meant "to
bedrock", right? The key I use is an openRouter API Key, so if that routes
differently in your PR, then that could be an issue as well. But my
assumption is that bedrock is just the grouping of all the different
models? But either way, using the same key I should get the same experience
between existing marketplace version and your PR version. — Reply to this
email directly, view it on GitHub <#60 (comment)
https://github.com/RooVetGit/Roo-Cline/pull/60#issuecomment-2546697657>,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGWMILCZDELXJXE3PNLSSUT2F43G7AVCNFSM6AAAAABTLSYLP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBWGY4TONRVG4
https://github.com/notifications/unsubscribe-auth/AGWMILCZDELXJXE3PNLSSUT2F43G7AVCNFSM6AAAAABTLSYLP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBWGY4TONRVG4
. You are receiving this because you were mentioned.Message ID: @.>
hmmm, ok
Could you go into details exactly how one would go about using the bedrock
service? We need to be able to test these changes...
However, I am still concerned that my current experience differs between
your PR and the currently available version when it comes to tracking
tokens and cost as the task progresses.
—
Reply to this email directly, view it on GitHub
https://github.com/RooVetGit/Roo-Cline/pull/60#issuecomment-2546755313,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AGWMILE67QOWWA4WBLWJSOL2F45RNAVCNFSM6AAAAABTLSYLP6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNBWG42TKMZRGM
.
You are receiving this because you were mentioned.Message ID:
@.***>


[Question]If a interface used memory cache the request.timing().responseEnd ‘s  value is always  same.

helllo，I find if a interface used memory cache then  I use page.reload() and collect the requests's info . then I find two question.
the first question is the interface used memory cache  ,I  view the interface  in devtools（ f12） , the time is 0ms.then view the request.timing().responseEnd in playwright .It shows dozens of milliseconds.
I'm confused why caused the dozens of milliseconds.

the second question is  if Used the page.reload and the interface used memory cache  . I find the value of request.timing().responseEnd is the same. No matter how many times the page is refreshed.

I sincerely look forward to your answer. thank you.

This is a bug, we cannot trust the regular network timing data when resource is served from memory cache as it contains some stale data (which is why it's the same on every reload). DevTools resets it to 0 in the UI when the request is served from memory. We should set requestStart/responseStart/responseEnd to -1 when the request is served from memory.


[PE-D][Tester D] App cannot check insanely big indexes

When large indexes are used for the edit and delete command, it outputs that the name cannot be found, which might confuse the user to change the name instead. Maybe use long/big integer instead, or check for integers above a certain big number?

Labels: type.FunctionalityBug severity.Low
original: alex-setyawan/ped#5

Edit EditCommandParser to check for index < MAX_INT


Audio length 1s

Is it okay to extract audio feature whose length is 1~3s? Its fbank shape is (139, 128)....^^. It means that n_frames = 139.

Yes, the AST model certainly supports 1s as our SpeechCommands recipe runs on 1s audios and achieves state-of-the-art performance.
If you use our training pipeline, you just need to change https://github.com/YuanGongND/ast/blob/97e57e7852809c6bc825c87a59f07635138cc43d/egs/speechcommands/run_sc.sh#L34 to your desired n_frames, in your case, 139. For optimal performance, you might want to set https://github.com/YuanGongND/ast/blob/97e57e7852809c6bc825c87a59f07635138cc43d/egs/speechcommands/run_sc.sh#L21 as True and do other tuning, please check the readme file for details.
-Yuan

If audio length is much shorter than 10s like 1s~3s, do i have to pretrain ast from scratch?

In my experience, audioset pretraining does not hurt the performance in almost all cases, so you can certainly have a try to set audiosetpretrain=True and imagenetpretrain=True like we did for the ESC-50 recipe. A finetuning stage is crucial for AST to achieve the optimal solution.
If you want to freeze the AST model and get the feature, it might be better to pad your input to 10s, I would suggest to use this inferernce script: https://colab.research.google.com/github/YuanGongND/ast/blob/master/colab/AST_Inference_Demo.ipynb, but instead of getting the last layer output (prediction logits), get the penultimate layer output as the feature. It should be a relatively easy modification and you don't need to worry about your input. The script loads audio and pad it to 10s.
-Yuan

How about your pretrained model? Your pretrained model also works well in short audio?

By audiosetpretrain=True, I meant our pretrained model.
For end-to-end fine-tuning, it works well for shorter audios, please see our ESC-50 recipe.
For freeze and feature extraction, I think padding it to longer audio is the best choice, please check the colab script.


Simple input signal loses focus after first character of input

Here's an example:
locNameSig :: Maybe String -> Signal HTML (Maybe String)
locNameSig nameMay = step nameMay do
  newName <- D.div' [
    D.h3' [D.text "Institution Name"]
  , D.input [P.unsafeTargetValue <$> P.onChange]
  ]
  pure $ locNameSig $ if newName == "" then Nothing else Just newName

After entering a single character, focus is undesirably lost. Some google searches suggest this may be because the widget is being re-rendered, which makes sense, but why only on the first key press? Subsequent entries into the input field work as expected, and in in all cases the output is always emitted as hoped.

Reference: https://github.com/erikras/redux-form/issues/1094

I'm not sure what I changed, nothing in the code above, so it must have been external. But it seems working now, so I'll close this for now.


Missing directory/actual files fonts/google/andada/files/*

Describe the bug
Missing files folder after NPM install and also no files folder in the repo
To Reproduce
Steps to reproduce the behavior:

See other repo font directories
or install via npm and take a look into node_modules/@fontsource/andada

Expected behavior
Install the actual fonts!

This is interesting.
Andada is deprecated so we can't track it on Google APIs anymore.
But we don't store the font files in the repository anymore for Git size reasons.
I think we need to setup a system to detect when Google deprecates a font and handle it to preserve the font files for future Fontsource updates, because this will continue on with new fonts that get deprecated. We'll have to figure that out sometime soon.
For now, you can use version 4.5.0 which has the font files or use it's successor Andada Pro as a replacement.

Thank you!

I'd like to keep this open as it is still a valid bug that needs to be fixed eventually :)


Prepare for the next release

crossbeam-epoch 0.9.4 -> 0.9.5

Fix UB in Pointable impl of [MaybeUninit<T>] (#694)
Support targets that do not have atomic CAS on stable Rust (#698)
Fix breakage with nightly feature due to rust-lang/rust#84510 (#692)

crossbeam-queue 0.3.1 -> 0.3.2

Support targets that do not have atomic CAS on stable Rust (#698)

crossbeam-utils 0.8.4 -> 0.8.5

Add AtomicCell::fetch_update (#704)
Support targets that do not have atomic CAS on stable Rust (#698)

crossbeam 0.8.0 -> 0.8.1

Support targets that do not have atomic CAS on stable Rust (#698)

Closes #702

bors r+


BLE optimizations

Please make sure these boxes are checked, before submitting a new PR.

[x] You ran the local CI checker with ./check.sh with no errors
[x] You reference which issue is being closed in the PR text (if applicable)
[x] You updated the OmniPaxos book (if applicable)

Changes
Updates leader election.
Other Changes
Also adds std::time::Duration parsing to TestConfig TOML.

The f9b54ca commit adds the NotAccepted message so that a stale leader will remove a follower that is promised to a greater ballot from its promised followers. This can reduce traffic but the stale leader will still resend prepare messages to the unpromised followers every resend-message tick.


No output to uart after sleep

Maybe, it is just an issue on macOS. When I am using PicoDemoSleep on the Mac (using minicom or any other app as serial monitor), there will be no output to the uart anymore (such as printf) after sleep is over. Seems as if uart has been disabled or changed due to sleep mode. The LED is turning on and off, though => program works as expected.

Dear ghubcoder,
Thanks for your comment.
I have used USB indeed and also tried the functionality to disable USB during sleep and then re-enable it after sleep, but without success.
And I have experienced the problem with the Pico hanging up even when I am using the Raspberry Pi Pico standalone, i.e., powered from battery.
In the meantime I‘ve written an example that includes a C++ class for easier integrating sleep functionality into own programs. See https://github.com/ms1963/SleepyPico
I have also opened an issue on the Pico-Extras-Repository which has not beenaddressed so far.

Same issue here. I've tested with USB and UART (Windows 10). USB stops working after sleep, though the LED keeps blinking for a short time until it hangs. UART works fine for a while but eventually the whole system hangs just like with USB. @ghubcoder can you confirm that altering CMakeLists.txt and removing print functions fixed this hanging issue? I can't really remove all prints as stdio is important for my project.

@RayNieport Try removing this line:
https://github.com/ghubcoder/PicoSleepDemo/blob/55b557218fcb22b67c441b362d1c8fae0c4a5605/main.c#L53
For me at least that stops it from hanging. Other wise it hangs after 3 sleep cycles.

Thanks a lot. I will try it.
Prof. Dr. Michael Stal, Auenstraße 34, D-80469 München
Chefredakteur/Editor-in-chief JavaSPEKTRUM
Professor @ University of Groningen

Am 13.08.2022 um 15:54 schrieb ghubcoder @.***>:

@RayNieport Try removing this line:
https://github.com/ghubcoder/PicoSleepDemo/blob/55b557218fcb22b67c441b362d1c8fae0c4a5605/main.c#L53
For me at least that stops it from hanging. Other wise it hangs after 3 sleep cycles.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you were mentioned.

Hello. I had the same problem.
It seems that the problem is that in the loop stdio_init_all()→stdio_usb_init()→irq_add_shared_handler(), I keep adding the collection of IRQ handlers without releasing them.
https://github.com/raspberrypi/pico-sdk/blob/6a7db34ff63345a7badec79ebea3aaef1712f374/src/rp2_common/hardware_irq/irq.c#LL215C12-L215C12
It may be correct to properly free these IRQ handlers before Deep Sleep, but as @ghubcoder says, simply removing stdio_init_all() in the loop will also avoid the crash.
However, in this case, the UART log will not work properly, so running stdio_uart_init() instead will cause the messages to be output properly.
(I'm using machine translation, so sorry if it contains strange English expressions.)


add a uniform prefix on database tables name

Gitea version (or commit ref):
Git version:
Operating system:
Database (use [x]):

[ ] PostgreSQL
[ ] MySQL
[ ] MSSQL
[ ] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[ ] No
[ ] Not relevant

Log gist:

Descriptio
Hello , I suggest to add A uniform prefix, such as gitea_, to the database table names. It will be usefull for user who want to use one datebase for many  application programs. and can prevent tables from being deleted by using other apps, thanks.
Screenshots

As of 1.12.0 you will be able to specify a database schema in PostgreSQL different from default. No work has been done for other databases ATM.

Hmm, This issue is forgoten in long time, please check it agin. Thanks @guillep2k
Now add more messages about this issue reason. That's Because some users use more than one apps in one database. I know it not a good idea, But if user do it, We will face a problem that other app will do change in gitea's table accidentally. if add prefix on table name. This can be prevented to some extent. I know it's a big work or even not necessary. so this issue can be closed if you like. Thanks

This is not difficult since xorm support that. We only need a PR.
x.SetTableMapper(names.NewPrefixMapper("gitea_", x.GetTableMapper()))

This is not difficult since xorm support that. We only need a PR.
x.SetTableMapper(names.NewPrefixMapper("gitea_", x.GetTableMapper()))

Hmm, but Don't forget we have use x.SQL or x.Exec in many places ... 😅
example:
https://github.com/go-gitea/gitea/blob/922a239079aefb78a05f1234f8883d8096f46c62/models/review.go#L375-L382

However I personally think that using the same database for different applications is bad practice, since all three engines (disregarding SQLite) support multiple databases for a single instance (and do have, like MSSQL's master, model, tempdb, etc.).
If two apps share the same "database", access permissions become more complicated, maintenance (backup, tuning) is more difficult, and restoring the database in case of data loss (e.g. a bad operation) forces to restore all the inter-dependent applications, even if only one of them was affected. I don't know what would be a good argument in favor of sharing a single database.

I know it, But many apps like discuz(pre_) , wordpress(wp_) have it, maybe it's want to label their apps's tables, or  just follow some unify name rules .... 🤔

We have to also create a wrap function
func realTablename("tablename") string {
    return prefix+"tablename"
}

ping, I will try it a few days later.


some linting cleanup

As more people become interested in helping grow this project, it would be a good idea to set some clear "requirements" for code style and linting.  I'd recommend that black be used for formatting, pylint and/or flake8 for linting, and potentially isort for sorting imports (with black compatibility mode).  Checks with all of these can be incorporated into the CI so that non-conforming code is rejected.  I can help with some of that setup, if you're interested in that, to make the libraries play nice together.  You could even include a sort of "build" script that would setup the virtual environment, install the dev dependencies, and could be run to do a code check locally to be run before committing.
Anyway, this PR contains some changes based on using pylint and flake8.  I disabled some errors that I don't suspect you'd want to be changed, but even with some messages disabled, this does not quiet fix all of them.  If you like these changes, then I'll plan to do a few others too.  Most of the changes in this PR boil down to one of:

Don't use str nor exit as variable names, which are built in python names
Removed unused variables, for example on a line like (fig, ax) = plt.subplots() where only the ax is used, I've replaced fig with _ which is the de facto standard to indicate that we don't care about that return value
Removed argument ax from setup_plot (not used in function)
Converted a handful of methods to use the staticmethod decorator since they didn't use self at all
Changed two of the plotting functions to return None, None if ax_supplied is true.  Best practice is that a function that returns something should always return something.
Simplified a number of if/else loops
Removed lists as default values to methods (dangerous if the function is called multiple times!)

And perhaps the biggest change here is that I've changed GeometryCleaner to be GeometryCleanerMixin for use as a mixin to the Geometry class.  The way clean_geometry was handled before was rather weird where the returned geometry object gets re-assigned back to self.  After this PR, the methods from the geometry cleaner could just be moved straight into the Geometry class, but perhaps it is nicer to keep it as a mixin to clearly differentiate the cleaning functions from the rest of the class.

This is great! Keen to take this forward and formalise.
I'm not super familiar with black or incorporating into the CI - do you need to specify some formatting rules (e.g. line widths etc.) and tell the CI what these rules are? And how is this information usually conveyed to potential contributors?
Happy to be taken to school on this one :)

This is great! Keen to take this forward and formalise.
I'm not super familiar with black or incorporating into the CI - do you need to specify some formatting rules (e.g. line widths etc.) and tell the CI what these rules are? And how is this information usually conveyed to potential contributors?
Happy to be taken to school on this one :)

My (limited) experience with black is that there is really nothing to configure or setup. It does it one way, and that's it. For libraries that I know of using it, they have a badge on their readme and a short blurb/link saying they use black, and whether CI will automatically do it.
There are some things in my initial testing that it didn't handle well, like long raw strings. For the most part it's just plug and play.

My (limited) experience with black is that there is really nothing to configure or setup. It does it one way, and that's it. For libraries that I know of using it, they have a badge on their readme and a short blurb/link saying they use black, and whether CI will automatically do it.
There are some things in my initial testing that it didn't handle well, like long raw strings. For the most part it's just plug and play.

black is intentionally very opinionated and has very few options as its intention is to minimize discussion on what the "best" formatting is and allow you to instead focus on the code itself.  I've used it extensively for every project and script I've written for years now and it is fantastic, in my opinion.  Admittedly, you'll fight it the first couple of times you use it since it may format something "not the way I want it"....but after you accept it, you get used to it and learn to really appreciate it.  Typically I have my editor to just format the code each time or save, or else regularly format it myself.  Then you can just kind of code and ignore most all of the formatting, then like black fix it all.
I personally do use two of the settings it does offer.  One is the line length (so 100 for this project) and another I always use is to "skip string normalization" as I personally dislike double quotes.  By default, it (generally) converts all single quotes to double quotes, but with the skip option it just leaves them as-is, so they stay single (') or double quoted quoted (") based on whatever you type in the code.
There is 1 time I've seen that it sometimes doesn't respect your line length, and that is with really long strings.  It tries to respect it, but it it cannot, then it lets the line be too long.  Image, for example, a string that is 150 characters long.  Obviously it cannot make that string fit within a 100 character length line.  So in this case, you can accept the long line length or you can manually wrap it yourself (what I do).  To manually wrap it, you can just use pythons implicit string concatenation within tuples.  The black project is in the early stages of implementing automatic string wrapping too, but it is actually a very tricky problem with many edge cases, so it isn't in their released versions as of now, but is available for early testing.
As for telling people about it (black and any other tools you choose to enforce, like flake8, etc), there are generally a few ways:

Often a project uses a badge at the top of the README as @Czarified mentioned.
Most projects will have a contribution guide, either as a section to the README or as a separate CONTRIBUTING.md (or rst) file that the README links to (common if the contribution guide is on the longer side).
Through enforcement in the CI, so that if a PR is opened that doesn't meet your code standards, the contributor knows right away as it fails CI.
You might have some sort of a "build" or "run" script (or even a Makefile) at the top level of the directory which is likely to catch the eye of most contributors --- and then this script can run the format and linting tools.
More experienced users will often check the settings in the project for various commonly used format/lint tools (pyproject.toml for black, others for other tools).

I'll take a stab at a pretty basic implementation with a few common tools and open a new PR and we can all discuss it further there, including if we want to keep or remove each tool.  And of course it can always change in the future too.

black is intentionally very opinionated and has very few options as its intention is to minimize discussion on what the "best" formatting is and allow you to instead focus on the code itself.  I've used it extensively for every project and script I've written for years now and it is fantastic, in my opinion.  Admittedly, you'll fight it the first couple of times you use it since it may format something "not the way I want it"....but after you accept it, you get used to it and learn to really appreciate it.  Typically I have my editor to just format the code each time or save, or else regularly format it myself.  Then you can just kind of code and ignore most all of the formatting, then like black fix it all.
I personally do use two of the settings it does offer.  One is the line length (so 100 for this project) and another I always use is to "skip string normalization" as I personally dislike double quotes.  By default, it (generally) converts all single quotes to double quotes, but with the skip option it just leaves them as-is, so they stay single (') or double quoted quoted (") based on whatever you type in the code.
There is 1 time I've seen that it sometimes doesn't respect your line length, and that is with really long strings.  It tries to respect it, but it it cannot, then it lets the line be too long.  Image, for example, a string that is 150 characters long.  Obviously it cannot make that string fit within a 100 character length line.  So in this case, you can accept the long line length or you can manually wrap it yourself (what I do).  To manually wrap it, you can just use pythons implicit string concatenation within tuples.  The black project is in the early stages of implementing automatic string wrapping too, but it is actually a very tricky problem with many edge cases, so it isn't in their released versions as of now, but is available for early testing.
As for telling people about it (black and any other tools you choose to enforce, like flake8, etc), there are generally a few ways:

Often a project uses a badge at the top of the README as @Czarified mentioned.
Most projects will have a contribution guide, either as a section to the README or as a separate CONTRIBUTING.md (or rst) file that the README links to (common if the contribution guide is on the longer side).
Through enforcement in the CI, so that if a PR is opened that doesn't meet your code standards, the contributor knows right away as it fails CI.
You might have some sort of a "build" or "run" script (or even a Makefile) at the top level of the directory which is likely to catch the eye of most contributors --- and then this script can run the format and linting tools.
More experienced users will often check the settings in the project for various commonly used format/lint tools (pyproject.toml for black, others for other tools).

I'll take a stab at a pretty basic implementation with a few common tools and open a new PR and we can all discuss it further there, including if we want to keep or remove each tool.  And of course it can always change in the future too.

Thanks for the rundown! If either of you are interested I think this would be an excellent addition to the project!
As this PR has already been merged (and buried a little bit in the gihub interface) I've created a new discussion at #52.

Thanks for the rundown! If either of you are interested I think this would be an excellent addition to the project!
As this PR has already been merged (and buried a little bit in the gihub interface) I've created a new discussion at #52.


Move to Hatch

Basically the title. See Hatch docs for more details.

Oh cool, thank you!
I haven't tried Hatch yet, but I guess this is a good opportunity. I will look at this and the Ruff PR at some point later this week :)

Oh cool, thank you!

No problemo!

I haven't tried Hatch yet, but I guess this is a good opportunity.

A good quickstart is:

hatch fmt runs ruff linter and formatter in one (use --formatter or --linter to run them separately)
hatch run types:check for type checking (PR incoming)
hatch test for tests (although there aren't any at the moment)

And the versioning docs might be worth reading separately 🙂

I will look at this and the Ruff PR at some point later this week :)

It's pretty tiny, even with the formatting/linting updates, but looks larger since it technically should target this branch, but thanks!


Task times are not shown when footer is hidden

I think. Happens with this config:
config:
  log-path: build.log
  collapse-on-completion: true
  show-summary-footer: false
  show-task-output: false
  # show-task-times: false

I expected the tasks to still be shown.

footer times and task times have been reworked in the last couple weeks and the issue has been fixed (somewhere)


I can´t launch mayncraft by java and bedrock

03:08:16.985
Reloading ResourceManager: Default
03:08:20.336

A fatal error has been detected by the Java Runtime Environment:

EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffd812b5970, pid=4728, tid=5412

JRE version: OpenJDK Runtime Environment Microsoft-32931 (17.0.3+7) (build 17.0.3+7-LTS)
Java VM: OpenJDK 64-Bit Server VM Microsoft-32931 (17.0.3+7-LTS, mixed mode, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
Problematic frame:
C  [glfw.dll+0x5970]

No core dump will be written. Minidumps are not enabled by default on client versions of Windows

An error report file with more information is saved as:
C:\Users\W10\AppData\Roaming.minecraft\hs_err_pid4728.log

If you would like to submit a bug report, please visit:
https://github.com/microsoft/openjdk/issues
The crash happened outside the Java Virtual Machine in native code.
See problematic frame for where to report the bug.

03:08:20.385
Process crashed with exit code 1

I hope you help me, since I just paid for that


info_for_paper_trail method on the ApplicationController inside the error

when i use devise and paper_trail in my project,
if info_for_paper_trail method on the ApplicationController inside the error
Version:
paper_trail-7.0.2
devise-4.3.0

Thanks for the bug report. Per our contributing guide please use
our bug report template.

Closing due to inactivity.


Button 6 (B button touched) isn't correctly set on right controller due to typo

https://github.com/GodotVR/godot_oculus/blob/master/src/ARVRInterface.cpp#L524
ovrTouch_Y should be ovrTouch_B to correctly register B button touches. Looks like it was an oversight when copying code for the right touch controller.
This seems to be affecting the godot-oculus asset on the Godot AssetLib.

Sorry for the late response, I had totally missed this issue. Should be fixed now in the master (Godot 3.1 only)

Should this issue be closed?

I think so :)


ctc_loss: W ./tensorflow/core/util/ctc/ctc_loss_calculator.h:499] No valid path found.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): Yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): MacOS
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 2.4.0
Python version: 3.6.7
Bazel version (if compiling from source):
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: CUDA 11.2
GPU model and memory: on CPU

Describe the current behavior
When using ctc_loss with dense tensor, it functions correctly. But when using it with sparse tensor, although the result is the right but it raised a warning saying:
W ./tensorflow/core/util/ctc/ctc_loss_calculator.h:499] No valid path found.

I'm not sure if this is a real bug or something else.
Describe the expected behavior
It shouldn't raise the warning.
Standalone code to reproduce the issue
import tensorflow as tf
label = [
	[1, 2, 1, 0, 0],
	[1, 1, 0, 0, 0],
	[1, 1, 1, 1, 1],
]
logits = [
	[[0.0, 1.0, 0.0], [0.0, 0.0, 1.0], [0.0, 1.0, 0.0], [1.0, 0.0, 0.0], [1.0, 0.0, 0.0]],
	[[0.0, 1.0, 0.0], [0.0, 1.0, 0.0], [1.0, 0.0, 0.0], [1.0, 0.0, 0.0], [1.0, 0.0, 0.0]],
	[[0.0, 1.0, 0.0], [0.0, 1.0, 0.0], [0.0, 1.0, 0.0], [0.0, 1.0, 0.0], [0.0, 1.0, 0.0]],
]
labels_length = [3, 2, 5]
logits_length = [5, 5, 5]
labels_tensor = tf.convert_to_tensor(label, dtype=tf.int32)
labels_tensor_sparse = tf.sparse.from_dense(labels_tensor)
logits_tensor = tf.convert_to_tensor(logits, dtype=tf.float32)

labels_length_tensor = tf.convert_to_tensor(labels_length, dtype=tf.int32)
logits_length_tensor = tf.convert_to_tensor(logits_length, dtype=tf.int32)

loss_dense = tf.nn.ctc_loss(labels_tensor, logits_tensor, labels_length_tensor, logits_length_tensor, logits_time_major=False)
print(loss_dense.numpy()[0])
loss_sparse = tf.nn.ctc_loss(labels_tensor_sparse, logits_tensor, None, logits_length_tensor, logits_time_major=False, blank_index=0)
print(loss_sparse.numpy()[0])

Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.

NVM, it's my mistake and understand the thing wrongly


Build and test Virtual Keyboard support for App Builder

Correctly integrate https://developer.chrome.com/apps/accessibilityFeatures#property-virtualKeyboard into App Builder, starting with kiosk apps.

According to this: https://code.google.com/p/chromium/codesearch#chromium/src/chrome/common/extensions/api/accessibility_features.json&q=virtualKeyboard file:js&sq=package:chromium&type=cs&l=48
The set() interface described in the developer documentation probably takes a Boolean Javascript value.

Setting of value in packaged apps still needs testing. I can do that. Jackson, can you add this to template app views that make sense? Also, do you think that a defaultValue of true is the right thing here?

The default should vary by app type. Event Calendars and catalogues, for instance, might not require any user text input unless search is enabled. Questionnaires will always require text input except in very specific circumstances (only checkboxes, for instance).


Use with ArrayField

The widgets provided for ArrayField in the admin do not handle the units in an elegant way. I would have expected a comma delimited list of values and a drop down for units afterward. Instead I get a comma delimited list of: value unit, value unit....

@ksmith0 do you still want to use an ArrayField?
I guess it will be hard to implement being a combination of different Widgets.
If not I would close this Issue - also if you not answering until next year.

I still think this would be a useful feature and make the user experience better.
I have no understanding of the underlying mechanics, just observing from the interface.

Is this still of interest?


Fix torch warnings

Hi @frankier, we've greatly changed the structure of the project and it caused conflicts we cannot resolve. I will take your suggestion but not merge this PR. Thanks!


Correct incorrect example in readme

The example usage of kubelet in the readme was incorrect.

Thank you @vdmkenny


Version2 development extract time bugfix

Minor changes to address the bug describes in issue #675.

This should be ready to merge. Please have a look at it when you can @bouweandela.


how to change transition based on slider value

I want to control pan and zoom effects through the slider. is it possible?

You can use the component in controlled mode like described in the Readme file.
You could then add change listeners to your sliders and change the state based on their current values.

Thank you @hatsuo for the answer. Im closing this but please reopen if you still have an issue.


Support Pagination for ClusterGroupMembership

To resolve #2492 .
Pagination:
API query: kubectl proxy &
curl "<k8s-apiserver>:8001/apis/controlplane.antrea.io/v1beta2/clustergroupmembers/<name>?page=<page#>&limit=<limit#>"
!! Notice the double quote, it's very important as curl would not interpret the & otherwise.
Page: the specific page to return in the sorted member list
Limit: max number of members to return per page
Response:
{
  "kind": "GroupMemberList",
  "apiVersion": "controlplane.antrea.tanzu.vmware.com/v1beta2",
  "metadata": {
    "name": <name>,
    "creationTimestamp": null
  },
  "effectiveMembers": [],
  "effectiveIPBlocks": null,
  "totalMembers": <number of members in entire list>,
  "currentPage": <page of response>,
}

Change:

Changed the Getter of cluster group membership querier to GetterWithOptions so it can take in more options.
Changed ClusterGroupMembers struct to include total members and current page info in the response.
Added PaginationGetOptions to both controlplane and v1beta2 types, registered in controlplane, v1beta2, and legacy APIs.
Changed parameterCodec from metav1 version to runtime scheme codec in pkg/apiserver/apiserver.go.
Added UT specifically for pagination Get.

Signed-off-by: Qiyue Yao<EMAIL_ADDRESS>

Codecov Report

Merging #3183 (67d4b37) into main (2ee6ad1) will decrease coverage by 20.20%.
The diff coverage is 64.00%.

@@             Coverage Diff             @@
##             main    #3183       +/-   ##
===========================================
- Coverage   60.59%   40.38%   -20.21%
===========================================
  Files         297      169      -128
  Lines       25434    21419     -4015
===========================================
- Hits        15411     8651     -6760
- Misses       8345    11924     +3579
+ Partials     1678      844      -834

Flag
Coverage Δ

kind-e2e-tests
?

unit-tests
40.38% <64.00%> (+0.04%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/apis/controlplane/types.go
0.00% <0.00%> (ø)

.../registry/networkpolicy/clustergroupmember/rest.go
74.35% <72.72%> (-13.88%)
:arrow_down:

pkg/ovs/openflow/default.go
0.00% <0.00%> (-100.00%)
:arrow_down:

pkg/util/runtime/runtime.go
0.00% <0.00%> (-100.00%)
:arrow_down:

pkg/ovs/openflow/logs.go
9.52% <0.00%> (-90.48%)
:arrow_down:

pkg/apis/controlplane/register.go
0.00% <0.00%> (-90.00%)
:arrow_down:

pkg/agent/controller/networkpolicy/reject.go
0.00% <0.00%> (-87.91%)
:arrow_down:

pkg/agent/nodeportlocal/k8s/annotations.go
0.00% <0.00%> (-83.88%)
:arrow_down:

pkg/agent/client.go
0.00% <0.00%> (-77.42%)
:arrow_down:

pkg/ovs/ovsconfig/ovs_client_linux.go
0.00% <0.00%> (-76.93%)
:arrow_down:

... and 237 more

LGTM overall, will let other reviewers take a look as well

/test-all

That doesn't have to be addressed in this PR, but it seems to me that we are doing a lot of repeated work every time a new page is requested: we build the entire slice of members, then sort it and then we index into it using the page number. I wonder if there is a way to implement it differently, while remaining stateless. Adding @tnqn to see if he has any insight on whether this is possible.

@antoninbas I thought about caching the result but it seems too complicated to associate queries and clean up cache. But If we assume the pagination will be used in a situation that only the first few pages will be queried quite often, maybe an optimization worth to do is sorting only the first pageNumber*pageLimit (say n) instead of the entire slice (say N) to reduce the runtime from O(NlogN) to O(nlogn)?

@tnqn and is the sorting actually required? Given that for this API, the user wouldn't expect any specific ordering of the results, there should be no need to sort as long as r.querier.GetGroupMembers(name) always returns members in the same order. Is that not the case?

That doesn't have to be addressed in this PR, but it seems to me that we are doing a lot of repeated work every time a new page is requested: we build the entire slice of members, then sort it and then we index into it using the page number. I wonder if there is a way to implement it differently, while remaining stateless. Adding @tnqn to see if he has any insight on whether this is possible.

@antoninbas I thought about caching the result but it seems too complicated to associate queries and clean up cache. But If we assume the pagination will be used in a situation that only the first few pages will be queried quite often, maybe an optimization worth to do is sorting only the first pageNumber*pageLimit (say n) instead of the entire slice (say N) to reduce the runtime from O(NlogN) to O(nlogn)?

Trying to understand the first pageNumber*pageLimit (say n), is it based on the original unsorted members? Then should there be another option for users to decide if they want continuous result (no member change) across queries?

@antoninbas I suppose the API will be used for listing members of a group in web application, it's possible that there are some updates to the members when user requests next page. If we don't sort it at all, the result of next page may be totally discontinuous from previous page even there is just one member removed/added to the group because the map may be resized. If we sort it, the pages could be continuous overall with a few duplicate or missing members, which is very common in list page (For example, if someone opens a new PR when I'm viewing page 1, I may still see the last PR of page 1 when I view page 2).
@qiyueyao I mean sorting the requested members only via something like heap sort. For example, when user requests page 1, pageLimit is 10, and the total members is 1000. In current implementation, the cost is sorting 1000 items => O(1000 * log1000). If we use heap sort, we only need to build a heap and get the least or largest 10 items => O(1000+10*log1000).
The runtime should be O(N+klogN), not O(klogk) I commented.

@antoninbas I suppose the API will be used for listing members of a group in web application, it's possible that there are some updates to the members when user requests next page. If we don't sort it at all, the result of next page may be totally discontinuous from previous page even there is just one member removed/added to the group because the map may be resized. If we sort it, the pages could be continuous overall with a few duplicate or missing members, which is very common in list page (For example, if someone opens a new PR when I'm viewing page 1, I may still see the last PR of page 1 when I view page 2).
@qiyueyao I mean sorting the requested members only via something like heap sort. For example, when user requests page 1, pageLimit is 10, and the total members is 1000. In current implementation, the cost is sorting 1000 items => O(1000 * log1000). If we use heap sort, we only need to build a heap and get the least or largest 10 items => O(1000+10*log1000). The runtime should be O(N+klogN), not O(klogk) I commented.

Understood. If we have the use case for heap sort, I could open an issue to track, and follow up with a new PR. Let me know if that works, thanks.

/test-all


Rework command line and IPC handling

The command line processing code is currently very verbose and like a spider web "everywhere". Once #7 has matured far enough, rewrite it in a more maintainable and extendable way.

Note: since we got a client-server model now with REST and Web Sockets this will be eventually solved with these technologies.


Rewrite da.fromfunction w/ da.blockwise

This simplifies the implementation of da.fromfunction by rewriting it in terms of da.blockwise. As a result da.fromfunction can take advantage of HLG by default. This also makes it more natural to feed into other HLG operations.

[ ] Closes #xxxx
[ ] Tests added / passed
[ ] Passes black dask / flake8 dask / isort dask

Also addresses the fromfunction task in issue ( https://github.com/dask/dask/issues/6791 )

cc @jrbourbeau

Planning on merging tomorrow if no comments


Fails when mirrorlist is present

Nice Plugin, thanks for creating it. I hit the following when setting this up today
Issue is two fold, one I'd consider a bug, the other an enhancement. If the mirrorlist attribute is set in the .repo file you get the exception below. Even if mirrorlists are not support, the plugin shouldn't barf.
It would be double awesome if mirrorlists were actually supported.
Traceback (most recent call last):
  File "/usr/bin/yum", line 29, in <module>
    yummain.user_main(sys.argv[1:], exit_code=True)
  File "/usr/share/yum-cli/yummain.py", line 285, in user_main
    errcode = main(args)
  File "/usr/share/yum-cli/yummain.py", line 136, in main
    result, resultmsgs = base.doCommands()
  File "/usr/share/yum-cli/cli.py", line 434, in doCommands
    self._getTs(needTsRemove)
  File "/usr/lib/python2.6/site-packages/yum/depsolve.py", line 99, in _getTs
    self._getTsInfo(remove_only)
  File "/usr/lib/python2.6/site-packages/yum/depsolve.py", line 110, in _getTsInfo
    pkgSack = self.pkgSack
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 887, in <lambda>
    pkgSack = property(fget=lambda self: self._getSacks(),
  File "/usr/lib/python2.6/site-packages/yum/__init__.py", line 669, in _getSacks
    self.repos.populateSack(which=repos)
  File "/usr/lib/python2.6/site-packages/yum/repos.py", line 308, in populateSack
    sack.populate(repo, mdtype, callback, cacheonly)
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 165, in populate
    if self._check_db_version(repo, mydbtype):
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 223, in _check_db_version
    return repo._check_db_version(mdtype)
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1256, in _check_db_version
    repoXML = self.repoXML
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1455, in <lambda>
    repoXML = property(fget=lambda self: self._getRepoXML(),
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1447, in _getRepoXML
    self._loadRepoXML(text=self)
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1437, in _loadRepoXML
    return self._groupLoadRepoXML(text, self._mdpolicy2mdtypes())
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1412, in _groupLoadRepoXML
    if self._commonLoadRepoXML(text):
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1230, in _commonLoadRepoXML
    result = self._getFileRepoXML(local, text)
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 1008, in _getFileRepoXML
    size=102400) # setting max size as 100K
  File "/usr/lib/python2.6/site-packages/yum/yumRepo.py", line 823, in _getFile
    result = self.grab.urlgrab(misc.to_utf8(relative), local,
  File "/usr/lib/yum-plugins/s3iam.py", line 95, in grab
    self.grabber = S3Grabber(self)
  File "/usr/lib/yum-plugins/s3iam.py", line 114, in __init__
    "'baseurl' value" % repo.id)

Hi,
The plugin now fails with:
/usr/lib/yum-plugins/s3iam.pyc: Unsupported attribute: mirrorlist.

Due to the highly-available nature of S3 and the purpose of this plugin, I see very little value in supporting mirrorlist.
Sincerely,
Mathias


[Feature Request] Option to Use Git Configured Editor as Default External Editor

Many users can be expected to have Git configured.  Thus, it is very likely that, if Git is configured, also a preferred editor is set.  With the git2 crate (MIT OR Apache-2.0), you can access this configuration and offer to use this editor as the default external editor.  If the configuration should not be accessible for some reason, e.g., the current working directory is not a Git repository, Git is not configured, or a preferred editor for Git is not set, the current implementation should be applied as fallback.  Thus, the configured Git editor would be an additional offer to the users.
For the implementation, first of all, run cargo add git2 --no-default-features to add the crate with minimal dependencies to the Rust manifest.  Then, to access the configuration, you can use the following function.
fn get_git_editor() -> Result<String, u8> {
  git2::Repository::open(".").map_or(Err(64),
    |r| {
      r.config().map_or(Err(69), |c| c.get_string("core.editor").map_or(Err(65), Ok))
    },
  )
}

The function returns you either the configured Git editor or an integer according to the sysexits header; the latter is meant to be a placeholder for your preferred return value in case of an error.  For the real Rust implementation of sysexits, you can use https://github.com/sorairolake/sysexits-rs (cargo add sysexits).
What do you think about this feature?

This could be useful for cross platform compatibility, since the current implementation has only UNIX in mind. I'll try to implement this and see how it would work

@kevinmatthes  I've found the method git2::Config::open_default() which opens the git global the configurations.
I think it's more useful this way since the users most likely set the editor globally and this way we are not tied to the current directory.
If you think that there are other benefits from using the current directory please let me know

@AmmarAbouZor
Thanks for pointing out the access to the global configuration!  I did only know about the local one, by now.
One thing that might be considerable is the ability to configure the preferred Git editor per repository.  This is especially useful for users who contribute to multiple text editors.  These users may have set their favourite editor in the global settings and altered their preferred editor per repository by configuring the respective editor the corresponding project is about.  This use case should rather not be underestimated as there are plenty of text editors, especially in the Rust ecosystem where implementing them is very easy; furthermore, I experienced myself that contributors to text editors usually participate in the development of multiple editors (or at least switch to another project when one should become discontinued whilst still the setting regarding the Git editor for the discontinued project is reasonable as the maintainer might decide to work on their project, again).
However, this is a rather exotic edge case as most users cannot be expected to contribute to more than one editor, if any.  So, it is up to you how you decide to take this scenario into account for the further development since the current implementation should fit for about 95% of the use cases, I guess.

For that scenario I think the best solution is to configure the editor from tui-journal configurations directly which gives the benefit of having the same experience using the app globally :+1:


refresh_signature lambda tries to access a non-existing collection and fails with 403

On -stage, v4.0.2, the refresh_signature lambda fails with the following error:
START RequestId: d05fc3a6-5ed2-11e8-8a09-6373fe64fc8c Version: $LATEST
Looking at /buckets/monitor/collections/changes:
Looking at /buckets/main-workspace/collections/test-leplatrem: GET /v1/buckets/main-workspace/collections/test-leplatrem - 403 403 - {'code': 403, 'errno': 121, 'error': 'Forbidden', 'message': 'This user cannot access this resource.'}: KintoException
Traceback (most recent call last):
  File "/var/task/aws_lambda.py", line 194, in refresh_signature
    collection_metadata = client.get_collection()['data']
  File "/var/task/kinto_http/__init__.py", line 595, in get_collection
    resp, _ = self.session.request('get', endpoint)
  File "/var/task/kinto_http/session.py", line 123, in request
    raise exception
kinto_http.exceptions.KintoException: GET /v1/buckets/main-workspace/collections/test-leplatrem - 403 403 - {'code': 403, 'errno': 121, 'error': 'Forbidden', 'message': 'This user cannot access this resource.'}

END RequestId: d05fc3a6-5ed2-11e8-8a09-6373fe64fc8c
REPORT RequestId: d05fc3a6-5ed2-11e8-8a09-6373fe64fc8c	Duration: 714.73 ms	Billed Duration: 800 ms 	Memory Size: 128 MB	Max Memory Used: 56 MB

However, the collection in the above error message doesn't exist. The collection exists in main and main-preview buckets, but not in main-workspace bucket.
$ curl -s --user "$userpass" https://kinto-writer.stage.mozaws.net/v1/buckets/main-workspace/collections/test-leplatrem | jq .
{
  "code": 404,
  "errno": 110,
  "error": "Not Found",
  "details": {
    "id": "test-leplatrem",
    "resource_name": "collection"
  }
}

Upon deleting the said collection from main and main-preview manually, the refresh_signature lambda is working again.
But still not sure why the lambda tries to access a non-existing collection in the first place, and not sure if this is a bug or expected behavior either.

I think we have to decide what to do when the source collection is deleted.
a) We could keep the preview and destination ones when the source collections is deleted, and skip collections with 404 status in the lambda. That would mean signature will never be refreshed on those ones (see https://github.com/Kinto/kinto-signer/issues/251)
b) We could delete the preview and destination ones when the source collection is deleted. This won't happen in production anyway.
c) We could give the write permission on the preview and destination collections to the creator of the source collection. This way the user that is allowed to delete the source collection is also allowed to delete the preview and destination manually.
I have a preference for a+c

We don't do self-service anymore. So deleting is now done via admin commands under control :)


Flying Chaos Monkey of Doom

Flies into a container and blows it up (kills it)
Similar to Netflix's Chaos Monkey

Very seriously the best github issue ever!


Clicking the IPIMS home button when a user is not logged leads to an error

Cause: user variable is not loaded, which means link to user home is invalid.
recommended fix to to handle the case in the respective controller, by redirecting to login page

Fixed. this. not the best. but it works. Genstats branch. UserController


Don't use bash arrays

FreeBSD apparently doesn't even have bash as an option by default (though it is available in ports), so it looks like I should figure out how to make csh/tcsh work.

Oops, didn't notice this before opening #5. You don't really have to support csh for FreeBSD - I don't think it's possible with a single script as csh has weird and completely incompatible syntax. Instead, you should just use #!/bin/sh and stick to POSIX shell features (e.g. no arrays, foo() {} instead function foo() {} etc.)

Arrays are the big one.  As I noted in #5, I did actually try to get it working without arrays but the quoting was never right, and eventually I gave up because bash fulfills all my use cases.  I'm happy to accept patches, but I don't plan on trying to get plain variables working again myself.
Some of the stuff that needs to work correctly:
./configure --prefix=/usr
./configure --prefix="/usr"
./configure --prefix="/usr" --libdir="/usr/lib64"
./configure --prefix="/usr" --libdir="/usr/lib64" CFLAGS="-Wall -Wextra -fsanitize=undefined"
./configure --prefix="/usr" --libdir="/usr/lib64" CFLAGS="-Wall -Wextra -fsanitize=undefined" CXXFLAGS="-Wall -Wextra -Wadd -DFOO=BAR"

Dropping the "function" keyword is trivial (TBH I didn't actually know that wasn't portable; AFAIK even dash supports it).  I'll change the title to "Don't use bash arrays", since that's really the issue.  If csh works that's great, but you're right, the real issue is POSIX-compliant shells.

Some of the stuff that needs to work correctly:

I don't see any problem with this. You get it as a single argument, you handle it as a single argument, (just don't forget to quote it everywhere).

I don't see any problem with this. You get it as a single argument, you handle it as a single argument, (just don't forget to quote it everywhere).

No you don't.  You get them as multiple arguments, and have to concatenate them into a string which the shell will then treat as multiple arguments when you pass them to cmake.  You don't want to pass values which may override per-project defaults unless they were actually provided to the configure script.
In theory it should be possible.  I don't remember what the issues were (it's been a while), but no matter what I tried there was too much quoting or too little.  I didn't need it to work with anything except for bash, so I just decided to use an array (which I find much more elegant anyways).

Oh, now I get it. It seems like you'll have to quote. Something like that (see also "Working with arrays" here):
quote() {
    echo "$1" | sed -e "s|'|'\\\\''|g; 1s/^/'/; \$s/\$/'/"
}

while [ $# != 0 ]; do
    case "$1" in
    --prefix=*)
        ARGS="$ARGS -DCMAKE_INSTALL_PREFIX=$(quote "${1#--prefix=}")"
        ;;
    --prefix)
        ARGS="$ARGS -DCMAKE_INSTALL_PREFIX=$(quote "$2")"
        ;;
    esac
    shift
done

eval "cmake $ARGS"

Also note a way of cutting of parameter name without calling cut. I've been told that this feature is POSIX compliant, but I'm not 100% sure. Though it's at least well supported by /bin/sh on FreeBSD.

Nice, thanks!  I've changed the script to use that code (as well as removing a few other trivial bashisms, such as "source" instead of ".").  I've only tested with dash, but AFAIK it should work everywhere…

It works for me, congrats! Couple more suggestions on code:

Everything passed to eval should be quoted. Eval interprets its arguments as a single string, so you actually have

eval "cmake ${TOP_SRCDIR} -DCMAKE_BUILD_TYPE=${BUILD_TYPE} -DCMAKE_INSTALL_PREFIX=${PREFIX} -DCMAKE_INSTALL_LIBDIR=${LIBDIR} ${CMAKE_ARGS}"

(and better to make it like that in code to make it apparent). You need to quote paths as well.

I've just realized that prefix cutting may be simplified even more: ${1#--prefix=} may be changed to ``${1#*=}```.

Everything passed to eval should be quoted. Eval interprets its arguments as a single string, so you actually have

Done, thanks.

I've just realized that prefix cutting may be simplified even more: ${1#--prefix=} may be changed to ${1#*=}`.

Ah, nice.  I already liked ${1#--foo=} a lot more than the old function, that is even better.  Huge improvemets, thanks!
Closing this now, thanks for all your help.  I really appreciate it.


HTMLSelectElement.options should be an HTMLOptionsCollection, not HTMLCollection

According to
https://developer.mozilla.org/en-US/docs/Web/API/HTMLSelectElement
the options property should be an HTMLOptionsCollection
https://developer.mozilla.org/en-US/docs/Web/API/HTMLOptionsCollection
Which is just like an HTMLCollection but all the options are HTMLOptionElement.
This allows code like
let select: HTMLSelectElement = ...;
console.log(select.options[0].value);

to work without needing to coerce select.options[0] to a HTMLOptionElement.
See also this (closed) issue for some background -- I think the type of this got a bit funky at one point due to a bug in Edge that was fixed: https://github.com/Microsoft/TypeScript/issues/1558

This is fixed already.
https://github.com/Microsoft/TypeScript/blob/906ce18b1b5c45374d2bb317f9c707e448c17b55/lib/lib.dom.d.ts#L6791


docs: References-anvil Reference - Examples - missing backtick in codeblock

References
|__ anvil Reference
|__ EXAMPLES
|__  Codeblock for point number 1  missing a backtick , that has been added, its not a major issue , just when copying the block directly, generates some errors.

@all-contributors add @m0ham3dx for maintenance


Create savremena.txt

Website - http://en.savremena-gimnazija.edu.rs/
Email proof (On the bottom of the main page) - https://i.imgur.com/RGCwYrI.png

@the-nick-dev Pull request merged. Please notice that free educational license for JetBrains tools is for high and higher schools students, i.e. those who are over 13 of age. Thank you!


Make host.docker.internal work with all WSL2 permutations and gitpod, fixes #4242, fixes #3772, fixes #3813

The Problem/Issue/Bug:

#4242
#3772
#3813
#3781
Gitpod hasn't been working right for some time due to something about how they bring up their docker network during prebuild, https://github.com/gitpod-io/gitpod/issues/9565

TODO

[x] Add automated testing of WSL2 with docker inside #3813
[x] Use the Docker 20.10 technique where possible
[ ] Improve TestDdevXdebugEnabled to cover more cases if possible
[x] Add a configuration option for when people are running IDE inside WSL2
[x] Allow an explicit IP address for XdebugIDELocation
[ ] Docs for unusual situations.

Resources

WSL2: The official Microsoft technique for finding out the Windows host IP is to look in WSL2's /etc/resolv.conf. That's what's needed for PhpStorm running on the Windows side. (Note that on machines that Docker Desktop has once been installed on, the host side has host.docker.internal in C:\Windows\system32\drivers\etc\hosts, and that host.docker.internal propagates to WSL2, solving this (if ddev doesn't overwrite it).
Gipod: @shaal points out that since Docker 20.10 the --add-host="host.docker.internal:host-gateway" approach has been available. The last time we worked on this, 20.10 was way too new to require of people, but now it's not. this article summarizes.

How this PR Solves The Problem:
Try to provide support for host.docker.internal in all the places:

For regular linux and Gitpod and WSL2 (IDE on WSL2) use the gateway host technique
For WSL2 with IDE on Windows, use the /etc/resolv.conf technique
Everywhere else, use the classic approach, mostly Docker Desktop provided.

Manual Testing Instructions:

[ ] Use xdebug on WSL2 with following matrix:

docker-ce installed in WSL2, PhpStorm on Windows
docker-ce installed in WSL2, PhpStorm inside WSL2
Docker Desktop integration, Phpstorm on Windows
Docker Desktop integration, Phpstorm inside WSL2
docker-ce installed in WSL2, vscode on Windows (use code . in WSL2 to launch
docker-ce installed in WSL2, vscode running inside wsl2

[ ] Linux Docker 20.10+  with PhpStorm

[ ] Windows with PhpStorm

[ ] macOS with PhpStorm

On most of these you know things are OK if ddev exec telnet host.docker.internal 9003 connects when the IDE is listening.
Automated Testing Overview:

Related Issue Link(s):
Release/Deployment notes:

[ ] This changes the required Docker version to 20.10.0+, which should work everywhere I know of. But this needs to be announced.

Issue: xDebug was not working in combination with DDEV, GITPOD and Phpstorm EAP
Installed the ddev-linux-amd64.zip from 'pull/4252' manualy in the GITPOD machine with:
wget https://nightly.link/drud/ddev/actions/artifacts/407640044.zip &&   unzip 407640044.zip &&   sudo chmod +x /usr/local/bin/ddev &&  sudo mv ddev /usr/local/bin/ddev
After install start DDEV again fixed the issue and phpstorm starts listening correct on 9003 from gitpod!


Replace configuration code with new config module

build-artifacts.py and system-tests.py duplicate configuration code. They should use the new config modules added in #16

https://github.com/nirs/oci/blob/857f949d17dcd6b3cd985abe1b55aa3c6444affc/build-artifacts.py#L18
https://github.com/nirs/oci/blob/857f949d17dcd6b3cd985abe1b55aa3c6444affc/system-tests.py#L25

@Funkabell do you want to take this?

@nirs yes good point, will do!


Allow Pramen-Py transformations to be at arbitrary folder

Originally reported by @MDobransky
Background
Currently, custom transformations are required to reside in the folder with the fixed name: 'transformations'.
This is good for the purpose of combining transformers from different packages.
But sometimes people would want to segregate dev and prod versions of packages by prefixes, say my-transformers-dev and my-transformers (prod). This type of segregation allows mixing and matching dev and prod versions of different transformers in one virtual env. it might be useful, for instance, to test specific dev transformers with other prod ones.
Observation
It's unintuitive that transformers from different packages overwrite each other when they have the same name.
Proposed solution
Allow the folder for scanning transformers to be configurable.
[To discuss] Ideas:

Globally, for a specific run, e.g. use only transformers defined in folder X.
Per package, keeping in mind how we want to mix and match them

Thanks for reporting it. What are the reasons that do not allow to install dev and prod transformers into separate python environments (with different pramen-py installations). If there is a need to access some prod transformers in dev environment, then why not just install them into the deve environment?


Added the rules button in 15 puzzle game.

Please let me know if i have to change anything or if i can proceed in a similar way with other games.

@RadhikaManohar We apologize for the inconvenience. We corrected the alignment after discovering a problem with the website last night, which may have caused confusion if you view the deployment preview of your PR. Please fork the updated code and add your code and make a new pr


app crash in example

in my ubuntu

I'm just download an compile example.Where is wrong?Please help me.

Hi! This code works with Theano as a backend, what is your backend?

@pplonski My Windows use Tensorflow backend,Ubuntu no backend installed.

This is a bunch of code to port Keras neural network model into pure C++.

This repo still need install backend?
I'm use my_nn_arch.json,my_nn_weights.h5,dumped.nnet under example directory,and did not run mnist_cnn_one_iteration.py.

OK, if you run on example data, then it should work. I'll try to reproduce and let you know. Thanks!

Thanks! :D

Hey! Sorry that it took so long, it is fixed now. Would you try it again?

It works well.Thank you!

You welcome!


Regarding Seelah's deity

It doesn't seem to be effecting anything at the moment (on a fresh / new game), seems to be primarily visual but when I look at Seelah's class ingame. It doesn't show she's selected Iomedae at 'creation'. Can this cause any problems or is it nothing to worry about? As her special abilities still list Iomedae.
Thank you

It's a UI bug, if you look at class features her faith is still their and should work as normal.
I will close this ticket for now but I do have to have a look at sorting the UI in the future just to make it look a bit nicer.


Fix V installation

@tobealive I do a quick fix for V installation problem - now it will install the latest commit from the main branch. I also improved your change so user don't need to change project name at all - it will be automatically retrieved from GitHub Actions variables

Very nice, especially the improvement with automatic retrieval!
What I've noticed is that it might be worth to still keep caching. The consecutive install steps are reduced to 1s then.

Since the installation doesn't take long and V is as you commented rapidly changing it's also okay without. Just wanted to leave this as a note.

Aaa, right. O wanted it to be done so quickly that I didn't think about that one possibility. I will do it rn

Cache key could be based on project version when v is more ripe.
key: ${{ runner.os }}-v-${{ hashFiles('**/v.mod') }}

Until then maybe something like:
echo "V_VERSION=$(v -v)" >> $GITHUB_ENV

And use the V_VERSION in the key.

Cache key could be based on project version when v is more ripe.
key: ${{ runner.os }}-v-${{ hashFiles('**/v.mod') }}

Until then maybe something like:
echo "V_VERSION=$(v -v)" >> $GITHUB_ENV

And use the V_VERSION in the key.

Nice idea with hashFiles('**/v.mod'), but I think for now I just stop on echo "V_VERSION=$(v -v)" >> $GITHUB_ENV where instead of v -v I will use ${{ steps.install-v.outputs.version }}

Hmm, I thought that V with check-last: true will have a version of commit, but it has just 0.3.3

Now it contains the commit ID, thanks @tobealive for the good help with that V_VERSION code ❤

Checking on the version that's installed despite check-last: true it's not latest commit. Looks fairly old:
https://api.github.com/repos/vlang/v/commits/d8167b8

Checking on the version that's installed despite check-last: true it's not latest commit. ~Looks fairly old~: https://api.github.com/repos/vlang/v/commits/d8167b8
Sorry all cool. Was confused with something else about the date.

I install the latest commit, just version output variable contains only X.Y.Z version. Anyway, this change merged

Yep ${{ steps.install-v.outputs.version }} is 0.3.3. v -v has the commit.

And good work. Thanks a lot!


Docker for gluten running on centos 8

Can gluten run on centos 8？ And if so,  is it possible to provide a docker image?

@zhejiangxiaomai @weiting-chen  Is Centos8 script ready now? if so let's add it to readme.

I will test gluten+velox on centos8 soon.  Velox already use Centos8 as a part of their own CI.

Sorry for late reply. I create a pr https://github.com/oap-project/gluten/pull/994  to add readme for centos8. And fix folly not found problem.

Centos8 is now running well.


Fix 933160-21 and 942500-1 tests broken due to invalid URIs

Hi CRS team. Thanks for the great product!
Here is a pull request to fix broken tests on Nginx. Let me know if it's better to open an issue first.
Issue
The regression test cases, 933160-21 and 942500-1, are failing on Nginx.
Reason
The uri parameters in tests cases 933160-21 and 942500-1 start with ?param= without the actual path such as /. This is an ill-formed path in terms of the HTTP protocol spec, because it lacks the absoluteURI, abs_path nor authority. (Link to the spec fyi.)
Thus, the generated requests are rejected by Nginx before WAF sees them, which is not an expected behavior.
JFYI, this happens on Envoy server as well.
Logs
Before my fix, Nginx rejects the requests by the test cases with the following log.
2021/07/27 11:47:43 [info] 23#23: *4 client sent invalid request while reading client request line, client: <IP_ADDRESS>, server: localhost, request: "POST ?x=eval%28chr%28112%29.chr%28104%29.chr%28112%29 HTTP/1.1"

With my fix, the test cases pass.

Hi @nullpo-head !
Thanks for your PR, will check it soon. Just remember sometimes we are precisely trying to break specs/rfcs :)

Hi @nullpo-head !
Thanks for your PR, will check it soon. Just remember sometimes we are precisely trying to break specs/rfcs :)

Thanks for your so quick response, fzipi!

Just remember sometimes we are precisely trying to break specs/rfcs :)

As for these tests, they are not intentionally violating RFCs. They are a test for php function and SQLi respectively, and the what they want to test are the values of their parameters, not an invalid HTTP path request. Thanks for the note!

@airween : You are usually keeping an eye on broken NGINX tests. This PR seems to fix something that is quite obviously broken. Could you take a look, please?

Sure, I'll check these soon.

Meeting decision August 2021: @airween will review this PR.

This PR seems to fix something that is quite obviously broken

Yes, it does, but I'm curious what your expectations are for Nginx's tests: it sounds like you're expecting all tests to pass on Nginx. However, in fact, 126 tests fail in Nginx (modec3-nginx), and this PR only fixes 2 of them. Do you normally expect all tests to pass in Nginx?
IIUC, to fix the all failing tests,

The CRS regression test has to introduce test skipping mechanism in order to skip Apache-specific tests
Rewrite CRS rules that use regular expressions that matches Apache-specific UTF-8 character escaping.
Rewrite CRS rules that use REQUEST_BODY, which don't work well with XML on ModSec3

Thanks for you comments. We know that some tests fail in nginx because of problems in the connector, and disimilar behavior patterns.
My idea was to use the feature available in go-ftw to skip well known failing tests, once we are happy with the actual testing pipeline and we declare it stable.

The idea is clearly for ModSec3 to pass all the tests. We know that the engine / NGINX platform has shortcomings and we keep a close eye on those. Your PR fixes things where our close eye did not see sharp enough. That's why I called @airween to join us here.
It's unfortunate that ModSec3 is lacking several years after they declared it production ready, but there is relatively little we can do outside of providing patches (what we've done in the past). The very thing we won't be doing is making CRS weaker in order to make ModSec3 pass 100% of the tests. CRS is meant to be the full functionality.

Hi @nullpo-head,
thanks again for your pull request. I did some research, now please let me summarize my ideas.
I've reviewed the RFC, and as I interpret, it allows the empty URI - but this is just my opinion, please check it someone else as well.
Even so I think in these cases, when the request can be adjusted for more valid test results, we can change it. There is only one small note from me: the first test's rule checks the REQUEST_FILENAME|ARGS_NAMES|ARGS, and the test sends the attack in ARGS_NAMES, not the REQUEST_FILENAME (in the current state it's an empty string, with your patch is a /). That's why I can say we should accept this PR.
My opinions for your other observations:

However, in fact, 126 tests fail on Nginx (modec3-nginx), and this PR only fixes 2 of them. Do you normally expect all tests to pass on Nginx?

you are right, but I'm not sure anybody will be more happier if there will be two less faulty test from the 126 :).

The CRS regression test has to introduce test skipping mechanism in order to skip Apache-specific tests

it's a good idea, and as @fzipi wrote above, our new regression test framework will know this feature

Rewrite CRS rules that use regular expressions that match Apache-specific UTF-8 character escapes.

sorry, could you help me to clarify which "Apache-specific UTF-8 character escapes" are you thinking about?

Rewrite CRS rules that use REQUEST_BODY, which don't work well with XML/JSON on ModSec3

Unfortunately I know what are you feeling - but sorry to say, this is a known libmodsecurity3 bug. There are two separated issues, we really hope that the developers will fix them soon.
Please note, that the reference shows: This variable is available only if the URLENCODED request body processor was used, which will occur by default when the application/x-www-form-urlencoded content type is detected, or if the use of the URLENCODED request body parser was forced.
Please accept our viewpoint, that we need a reference system - not only for the mod_security2 (and Apache) users, but all of our customers.
Summed up so far: we can merge this PR without any risk.

@fzipi

My idea was to use the feature available in go-ftw to skip well known failing tests, once we are happy with the actual testing pipeline and we declare it stable.

I didn't realize that there's go version. Faster and supporting more features. It sounds awesome! I feel it may be even nicer if we could maintain testoverride per test file basis. My PR, which dune already mentioned you, might help, though this would be out of the scope of this PR of fixing failing tests.

The idea is clearly for ModSec3 to pass all the tests

Thanks for clarification, @dune73! It sounds awesome.

The very thing we won't be doing is making CRS weaker in order to make ModSec3 pass 100% of the tests. CRS is meant to be the full functionality.

Yes, I understand it and I didn't mean it, either. I just wondered if the status of Nginx tests is tracked, but that was a completely unnecessary concern. Thanks for your great work!

I really appreciate the detailed discussion, @airween.

I've reviewed the RFC, and as I interpret, it allows the empty URI

we should accept this PR.

Thanks! It's of course OK to merge this PR anytime. As for the RFC, thanks for taking your time to check it and I'm sorry if I was wrong. It seems that another section says abs_path in a request cannot be empty, but I'm not a HTTP lawyer and that would be a bit off-topic, so no problem.

sorry, could you help me to clarify which "Apache-specific UTF-8 character escapes" are you thinking about?

I'm talking about the regular expression of the rule 920460 and 941330, which seems to have too-many backslashes. It seems to depend on the fact that Apaches do escaping like \\xffff or \\uffff, instead of \xffff or \uffff I'll open another issue on that if I confirm that.

this is a known libmodsecurity3 bug

Oops, I didn't notice that the guy in the discussion was you. Thanks so much for your effort on that.

hi @nullpo-head,

I really appreciate the detailed discussion, @airween.

thanks :)

Thanks! It's of course OK to merge this PR anytime. As for the RFC, thanks for taking your time to check it and I'm sorry if I was wrong. It seems that another section says abs_path in a request cannot be empty, but I'm not a HTTP lawyer and that would be a bit off-topic, so no problem.

sure, that's why I asked someone to review that part. No problem.

I'm talking about the regular expression of the rule 920460 and 941330, which seems to have too-many backslashes. It seems to depend on the fact that Apaches do escaping like \\xffff or \\uffff, instead of \xffff or \uffff I'll open another issue on that if I confirm that.

oh, I see - sorry to say, but this is also a know libmodsecurity3 bug (if I understand you correctly). Here is a small summary, how the escape char handling works.

Oops, I didn't notice that the guy in the discussion was you. Thanks so much for your effort on that.

no problem :). I've sent a very ugly PR to fix that, but I was just curious - that's not a real fix. Rather I devoted it to start a discussion.

Hey guys, I only skimmed through your conversation above. Can you give me a status report and a confirmation this PR is ready to be merged?

@dune73 Hi, we were discussing another topic, but I think the conclusion of this PR itself is this PR can be merged.

Agreed, merging 😄


Evaluate clustering with more batches?

Maybe the fact that we currently only take 10 batches is an explanation for why the clustering metrics seem so unsteady?

Also see #142


Add tox support for running tests and linting

What is the expected enhancement?
Tox is a nice tool for automating testing: https://tox.readthedocs.io/en/latest/ It enables automatically creating isolated environments for running tests by building a venv with all the requirements and then run test commands. In my experience it's real usefulness just comes from repeatability though. It makes it much easier for developers to run tests the same way as CI does without having to remember what's required to setup specific environments for running different types of listing. Adding a tox.ini for running pylint, and different testing configurations will make it much easier to verify code changes work before people submit PRs. Since it'll like be reduced to a single 3 letter command.
Right now I think the only blocker for this though is that by default we don't package a setup.py in the repo and the setup phase of tox expects that it can run pip/setup.py to install the package in a venv without anything extra.

For examples of other projects where this is used you can take a look at:

https://github.com/home-assistant/home-assistant
https://github.com/openstack/nova/
https://github.com/pytest-dev/pytest
https://github.com/PyCQA/pylint

All you need to do to run tests and style checks for all those projects is clone the repo and run tox


2.1.0-rc1: running second build with /scripts/eosio_build.sh fails

On 2.1.0-rc1
Using build script /scripts/eosio_build.sh -s EOS -P -y -i /local/lib/eosio-2.1
The first time I ran a build, it was fine (and it built all the libraries)
Next time I run the build, the libraries are already there.  But the build fails.
Current branch: HEAD
EOSIO will be installed to: /local/lib/eosio-2.1
EOSIO has already been installed into /local/lib/eosio-2.1... It's suggested that you eosio_uninstall.sh before re-running this script.

=====================================================================================
======================= Starting EOSIO Dependency Install ===========================
Architecture: Linux
OS name: Ubuntu
OS Version: 18.04
CPU cores: 4
Physical Memory: 65G
Disk space total: 3602G
Disk space available: 2910G
[Ensuring package dependencies]
 - git ok
 - make ok
 - bzip2 ok
 - automake ok
 - libbz2-dev ok
 - libssl-dev ok
 - doxygen ok
 - graphviz ok
 - libgmp3-dev ok
 - autotools-dev ok
 - python2.7 ok
 - python2.7-dev ok
 - python3 ok
 - python3-dev ok
 - autoconf ok
 - libtool ok
 - curl ok
 - zlib1g-dev ok
 - sudo ok
 - ruby ok
 - libusb-1.0-0-dev ok
 - libcurl4-gnutls-dev ok
 - pkg-config ok
 - patch ok
 - lsb-release ok
 - gnupg ok
 - No required package dependencies to install.

[Ensuring CMAKE installation]
 - CMAKE found @ /usr/bin/cmake.

[Ensuring Clang support]
 - Clang 10 found @ /local/lib/eosio-2.1/opt/clang10

[Ensuring LLVM support]
LLVM_ROOT (/local/lib/eosio-2.1/opt/llvm) already exists!
[Ensuring Boost 1.72.0 library installation]
 - Boost library found with correct version @ /local/lib/eosio-2.1/src/boost_1_72_0

========================================================================
======================= Starting EOSIO Build ===========================
Re-run cmake no build system arguments
-- The C compiler identification is Clang 10.0.0
-- The CXX compiler identification is Clang 10.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Boost 1.67 found.
-- Found Boost components:
   program_options;unit_test_framework
-- Configuring EOSIO on Linux
-- Using 'EOS' as CORE symbol name
-- Using 'EOS6MRyAjQq8ud7hVNYcfnVPJqcVpscN5So8BhtHuGYqET5GDW5CV' as public key for 'eosio' account
-- Found GMP: /usr/lib/x86_64-linux-gnu/libgmp.so
-- Configuring fc to build on Unix/Apple
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - not found
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Found OpenSSL: /usr/lib/x86_64-linux-gnu/libcrypto.so (found version "1.1.1")
-- Looking for pthread_setname_np
-- Looking for pthread_setname_np - found
-- Boost 1.66 found.
-- Found Boost components:
   date_time;filesystem;chrono;unit_test_framework;iostreams
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.11")
-- zlib found
-- Configuring Builtins
-- Configuring SoftFloat
-- Boost 1.57 found.
-- Found Boost components:
   filesystem;unit_test_framework
-- Configuring ChainBase on Linux
-- Boost 1.60 found.
-- Found Boost components:
   filesystem;program_options
-- Configuring AppBase on Linux
-- Found Git: /usr/bin/git (found version "2.17.1")
-- The ASM compiler identification is Clang
-- Found assembler: /local/lib/eosio-2.1/opt/clang10/bin/clang
-- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER
-- Performing Test HAVE_OMIT_LEAF_FRAME_POINTER - Success
-- Performing Test HAVE_SSE42
-- Performing Test HAVE_SSE42 - Success
-- Performing Test HAVE_THREAD_LOCAL
-- Performing Test HAVE_THREAD_LOCAL - Success
-- Enabling RTTI in Debug builds only (default)
-- Performing Test HAVE_FALLOCATE
-- Performing Test HAVE_FALLOCATE - Success
-- Performing Test HAVE_SYNC_FILE_RANGE_WRITE
-- Performing Test HAVE_SYNC_FILE_RANGE_WRITE - Success
-- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP
-- Performing Test HAVE_PTHREAD_MUTEX_ADAPTIVE_NP - Success
-- Looking for malloc_usable_size
-- Looking for malloc_usable_size - found
-- Looking for sched_getcpu
-- Looking for sched_getcpu - found
-- JNI library is disabled
-- Building eos-vm v1.0.0-rc1...
-- Installing...
-- Starting yubihsm configuration...
Install targets
Libraries /local/lib/eosio-2.1/lib
Includes /local/lib/eosio-2.1/include
Binaries /local/lib/eosio-2.1/bin
Manuals /local/lib/eosio-2.1/share/man
Pkg-config /local/lib/eosio-2.1/share/pkgconfig
not win32
-- Looking for memset_s
-- Looking for memset_s - not found
-- Looking for explicit_bzero
-- Looking for explicit_bzero - not found
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.29.1")
-- Checking for one of the modules 'libcrypto'
-- Checking for one of the modules 'libcurl'
-- Checking for one of the modules 'libusb-1.0'
-- yubihsm configuration complete
-- Checking for module 'libpq>=10.0'
--   Found libpq, version 13.1
-- Checking for module 'libpqxx>=6.0'
--   Found libpqxx, version 7.2.1
CMake Error at plugins/blockvault_client_plugin/CMakeLists.txt:25 (target_include_directories):
  Cannot specify include directories for imported target "pqxx".

CMake Error at plugins/blockvault_client_plugin/CMakeLists.txt:26 (target_link_libraries):
  Cannot specify link libraries for target "pqxx" which is not built by this
  project.

-- Configuring incomplete, errors occurred!

Actually this is "cmake" issue.  When using the system cmake, the build fails.

Actually this is "cmake" issue.  When using the system cmake, the build fails.

Actually this is "cmake" issue. When using the system cmake, the build fails.

Which cmake version?

Actually this is "cmake" issue. When using the system cmake, the build fails.

Which cmake version?

cmake from ubuntu: 3.10.2-1ubuntu<IP_ADDRESS>

cmake from ubuntu: 3.10.2-1ubuntu<IP_ADDRESS>

Fixed in rc3


Worker nodes shouldn't get updated before etcd/controlplane nodes

https://github.com/rancher/rancher/issues/23777

This has always been the behavior for RKE clusters, and logic hasn't changed as a part of the zero downtime feature. Closing it


Added support for private profiles

📑 Description
Implements #35 by optionally using OAuth.
✅ Checks

[ ] code: My pull request adheres to the code style of this project (I ran cargo fmt.)
[ ] docs: Added / updated (Which docs?)
[ ] tests: All the tests have passed (Which tests? I have tested this PR with my own private trakt profile.)

ℹ Additional Information
This PR adds the following entries to credentials.ini:
[Trakt API]
...
enabledOAuth = false
traktClientSecret =
OAuthAccessToken =
OAuthRefreshToken =
OAuthRefreshTokenExpiresAt =
...

If the user wants to enable OAuth, it is expected that they set enabledOAuth = true and traktClientSecret = <trakt client secret>, otherwise they can be left as is. The three other fields are set programmatically.
Application flow for using OAuth for the first time:

Set both aforementioned values.
Log into the automatically opened trakt OAuth website.
Paste the displayed OAuth code into the command line.
The other tokens are generated automatically and all the following starts of discrakt do not need any interaction.

I haven't found a nice way to write the generated tokens back to credentials.ini. The straightforward way to loading the whole config, changing the token entries and writing the whole config back to credentials.ini results in "ugly" formatting. Maybe writing them to the file "manually" would be better (but somewhat annoying to implement)?
The order of the entries in credentials.ini is now kept the same when writing the OAuth tokens back to the file (4336736). The only thing left regarding "ugly" formatting is that the spaces around the equal signs are removed when writing the values back to the file, but I haven't found a fix for that.

Sorry I didn't notice this PR until now! I'll try to review it tomorrow!

Thanks for the PR and sorry for the late review, merging in!


Do you have android client support quic-go server?

I want develop an android app use QUIC,Can you tell me which android client can use and compatibility this mvfst server?thank you very much.

Unfortunately, I don't have any information about android clients.