Avoid duplicate sorting in KeywordField#newSetQuery

Description
Term sorting can be a significant amount of work with a large number of terms. This change allows KeywordField#newSetQuery to do that work once, instead of TermInSetQuery and SortedSetDocValuesSetQuery duplicating the effort (and memory footprint as well).

I'm not sure how i feel about exposing PrefixCodedTerms in public apis. This really seems like an over-optimization and causes clutter. Maybe we can get @uschindler thoughts and consider using streams internally and avoid messing with PrefixCodedTerms, i think its possible to implement the KeywordField#newSetQuery so that stream is only sorted once.

Actually instead of taking a collection or array of BytesRef, we could make the ctor take a (Stream<BytesRef> stream) and consume the stream by calling: stream().sorted().distinct().doOurStuffByForEach().
The cool trick is that streams have the markers for "distinct" and "sorted" internally. So sorted() and distinct() do nothing if the stream is already marked using those flags. If not, the method returns a new Stream which sorts and has the correct flags. You should always have it in order "first sort, then distinct" (see some comments in JDK issues, it is no longer mandatory for optimal perf but always better to first sort then dedup).
The array ctor would just call Stream.of(array). This has no defined order and dedup, so the ctor taking the stream would sort for us. If we have a sorted list, we can trick the stream API to set the sorted and deduped flag with a helper method using spliterator of correct size and correct flags.
At end the ctor taking stream is plain simple and requests a sorted, distinct stream and it is up to the caller to have it presorted.

Of course it would still be needed to check performance of huge sets, but I think the stream overhead is so highly optimized in recent JDKs, so theres no issue (especially for large collections and many terms it is better memory-wise to not copy everything into an array and sort it). The user could also make the stream parallel :-)

Just to make sure I understand the suggestion here, it sounds like the idea would be to use a Stream<BytesRef> in place of PrefixCodedTerms both in TermInSetQuery and SortedSetDocValuesSetQuery right? And these streams would be backed by the same BytesRef[] provided to KeywordField#newSetQuery? I think there are a couple challenges with that (but maybe I'm misunderstanding the suggestion completely):

Wouldn't we still be sorting the data twice since we would need to provide separate streams to each query? Or I suppose we could maybe share a single stream across the queries if we knew for sure that only one would consume it?
Within each query implementation, we may end up iterating over the terms multiple times (e.g., if someone did a toString, visit, create a weight, etc.). I don't have a whole lot of experience using streams, but my understanding is they provide a "single consumption" model. So I'm not sure how this would work?

I suspect I'm misunderstanding the suggestion, so just trying to clarify and figure out where I've gotten confused. Thanks for the input! I like the idea, I'm just trying to figure out how to make it workable.

Thanks for looking into this @gsmiller I agree we should avoid computing the prefix terms twice. I like the approach of having a private ctor that takes prefix coded terms and using it from the KeywordField#newSetQuery method. As far as streams are concerned, Uwe's arguments are appealing but I have a (subjective) distaste of the Stream API which makes me prefer avoiding to use them in our public APIs. :)

To avoid leaking internal implementation details of the two query implementations, I've taken a different approach that doesn't expose the fact that we're internally representing terms as prefix-coded data.
This new approach just avoids duplicate sorting/cloning, but the prefix encoding is still done twice. I did some informal "benchmarking" of the two approaches with a simple unit test (see below), and the sorting/cloning of values is the most impactful bit of work to save. The duplicate prefix encoding didn't really register in the benchmarks (i.e., this approach and the previous approach were essentially the same).
To make it a "fair fight" with the current version on main, I fixed an issue in the current implementation where the values are not cloned before being passed to the SortedSetDocValuesSetQuery ctor. The current code on main is sorting the provided values array in place, which probably isn't the right thing to do (we correctly clone everywhere else we use SortedSetDocValuesSetQuery).
With the cloning in place, this PR cuts the "benchmark" results roughly in half (from 134ms to 68ms on my MacBook).
Here's my simple "benchmark":
  public void testSortPerformance() {
    int len = 50000;
    BytesRef[] terms = new BytesRef[len];
    for (int i = 0; i < len; i++) {
      String s = TestUtil.randomSimpleString(random(), 10, 20);
      terms[i] = new BytesRef(s);
    }

    int iters = 300;
    for (int i = 0; i < iters; i++) {
      KeywordField.newSetQuery("foo", terms);
    }

    long minTime = Long.MAX_VALUE;
    for (int i = 0; i < iters; i++) {
      long t0 = System.nanoTime();
      KeywordField.newSetQuery("foo", terms);
      minTime = Math.min(minTime, System.nanoTime() - t0);
    }

    System.err.println("Time: " + minTime / 1_000_000);
  }

The downside of this approach is that we're duplicating the memory footprint of the prefix encoded terms. I think that's an OK trade-off for now though to avoid exposing implementation details of TermInSetQuery, since there's no way to reduce the visibility of that ctor (we'd have to just tag it @lucene.internal).

I still don't understand why we are adding all this special logic to PrefixCodedTerms, to avoid a single Arrays.sort? The sorting only happens once in the query, its not like it takes milliseconds or anything to sort a damn array!
Need to understand the use-case here. I suspect its a severe database-abuse case. I have to push back on these since lucene is a search engine.

btw, latest commit is slightly better as at least it does not expose PrefixCodedTerms in a public API. But it brings java Collections into the picture, which seems not great versus using simple arrays.

@rmuir The use case where I've seen this in the wild has to to with allow/deny lists. We have some use-cases where we only want to match documents that exist in some allow-list. That allow-list can be quite large (potentially tens of thousands), but many of the terms aren't present in a given index we're searching. We use the bloom filter codec to efficiently drop terms not present. So we have a large number of terms we need to initialize our TermInSetQuery with, but a much smaller number of them actually end up term-seeking, etc., so the sorting actually appears to dominate when we've profiled.
I've "redacted" a bunch of this flame chart since this was on an internal system, but you can see how long we're spending sorting terms vs. everything else here:

Deny-listing can obviously have the same issue. I believe TermInSetQuery was originally created to handle deny-lists in tinder search where there can be tens of thousands of "swipe left" profiles that need to be excluded from results?

But it brings java Collections into the picture, which seems not great versus using simple arrays.

I don't love that either. An alternative could be to just require the ctor to be invoked with a pre-sorted array and not mess about with array copying and sorting. It would of course need to be a non-back compatible change released in 10, but that's OK.

Hi,
I fully agree with Robert here.  Yes if the array is large, the sort takes some time, sorry. But what does this have to do with PrefixCodedTerms? Why is Elasticsearch using this private API?
I would agree if we can internally have some special-case ctor that is package private (or it is shielded by using some shared-secrets approach if it needs to work cross-package), which allows to "reuse" an already existing PrefixCodedTerms instance internally when you build a TermInSetQuery from some case like building a TermInSetQuery for the joins module from a Term list. But PrefixCodedTerms should never ever be part of public APIs. If you want it "correct" use Java 8 streams API, although Adrien does not like it, but it is the "correct way" to do this kind of stuff.
If Elasticsearch needs to do some internal stuff and can't live with a (cheap) resort of an array or using java.util.Stream (if it is already sorted, TimSort will just linear scan through the array), it should fork the query as this is a special case and does not need to be supported in Lucene. All APIs are public, so you could make a subclass/fork  of TermInSetQuery that relies on some Elasticsearch-internal array container clsses that have a defined sort order. It is not Lucene's task to add support in our public APIs for some misuse of Lucene (and those lists of terms for access rights is a misuse of Lucene: it is a search engine and not a US NASA-for-travel-to-mars certified FIPS-0815 foobar bullshit with high security compliant and unbreakable datastore with strict user access checks on millions of documents).
Tell your customers that this is slower if you apply crappy access rights that should never have been in a serach engine!
Keep in mind: In any public API we have for sure to either always sort it or alterantively if we require a presorted array, we have to actually check this. If one passes an array in a public parameter and the defintion says it has to be presorted: WE MUST CHECK THIS!. If one would pass an unsorted array we would need to throw exception, so we need to check correct order as part of param validation. But as checking that the array is in correct order is as expensive as just sorting it, we should simply sort it. There's no discussion possible!

Let me try to clarify a couple things:
First, this is not an Elasticsearch use-case. Probably not all the important, but this is for an Amazon Product Search case, where we use Lucene directly. We're not applying access right, etc. We have use-cases where we need to evaluate allow-lists of products.
Yes, we can fork all these queries. But this looks like an opportunity to avoid cloning and sorting an array multiple times. If both of these underlying queries need sorted terms, it seems like it would be nice to avoid it.
Yes, a public constructor that requires sorted data should check it. The current proposal is doing this by leveraging the class type of the collection.
I also agree with not exposing the prefix-encoded implementation detail in the constructor, which I've also addressed in the latest revision.
It seems like the pushback here amounts to, 1) not wanting to mess with collections instead of arrays, and 2) generally feeling like this is an unnecessary complication for a non-general use-case. I personally disagree on both points. Sure, arrays are better in a number of ways, but this seems like a reasonable use of collections to "mark" sortedness as part of the type contract. While I also recognize that large cardinality term cases maybe aren't the "norm," we could extend that argument to say we shouldn't both with TermInSetQuery at all. BooleanQuery is just fine with few terms. I could create a slippery slope argument pretty quickly that says, "no more term in set at all!" I don't like what's at the bottom of that slope.
So I disagree with the pushback (at least in the current revision; I fully understand the pushback around not exposing prefix encoding details). But I don't disagree so strongly that I'll keep fighting for this if folks are unhappy with it. I'll do one more pass to see if I can leverage TimSort's efficient implementation on pre-sorted data. If that pans out, maybe it's enough to just sort the terms in KeywordField before passing them along. The underlying queries can still do their own sorting, which will just be a no-op. We'll still do some unnecessary array cloning, but maybe it doesn't matter.
Anyway, I appreciate the feedback and the discussion!

Sorry I thought this is Elasticsearch, because generally I get like daiky people asking how to hide their documents from users. At end they are always complaining that their SQL IN like queries are slow. This is why I always get angry and try to explain to people that they need to nenormalize their data. Instead of having huge lists of documents a user may see, it is better to instead index a multivalued field "users that can see this document". This always causes problems with those people that want to add new users all the time and need to therefore reindex everything. My general rule I tell them is always: index the user information in your weekly reindex and for those new ones to the heavy "SQL IN" query. But for the users that have their document they can see already reibdexes you can just query for a single term (their username".
So you may understand that I get angry about that at some point.  You need to always so some mixed approach. Denormalize as much as possible, and if not possible or partially use explicit IN queries. But really avoid that and use the inverted index for what it was invented.

I disagree with Robert who says "only arrays". I agree with you we can also allow to pass collections. But only when we do it as proposed before:
We can have 2 ctors, one with Collection<BytesRef> and one with BytesRef[]. But both should call a third hidden ctor taking Stream<BytesRef>.
This internal implementation would call: stream.sorted() (and possibly also .distinct()) and just operate on the stream. If you pass in a SortedSet (e.g. TreeSet) the sorted and distinct calls will be no-ops. It will not sort the stuff again, IF (and only IF) the comparator of the Treeset/sortedset is exactly the one we use. So it is 100% type safe.
I hope Adrien understand that this is the best way to avoid duplicate sorting:
public TermInSetQuery(String field, BytesRef[] terms) {
  this(field, Arrays.stream(terms);
}

public TermInSetQuery(String field, Collection<BytesRef> terms) {
  this(field, terms.stream();
}

private TermInSetQuery(String field, Stream<BytesRef> stream) {
  super(field); // and so on
  stream.sorted().distinct().forEachOrdered(term -> {
     // process the terms coming in natural order
  });
}

This would be my favorite way to implement this query. I don't think Adrien's mental problem with streams is an argument without a benchmark.

I disagree with Robert who says "only arrays". I agree with you we can also allow to pass collections. But only when we do it as proposed before:

Well, the newSetQuery() takes an array, so i don't like adding also collections also, unnecessarily. But especially i don't like complexity added with collections into PrefixCodedTerms which has nothing to do with arrays nor collections at all and is just a little part of the indexer becoming made more complicated by these horrible database-join-queries :(

Yes, a public constructor that requires sorted data should check it. The current proposal is doing this by leveraging the class type of the collection.

To me this is an AWFUL lot of complexity. Yes, you are just "marking" something as sorted, but java collections are just as bad to me as java streams are to adrien. Please, lets keep such stuff away from PrefixCodedTerms and the indexer.
If we can't implement TermInSetQuery without mucking up the indexer here, then maybe indeed we should just tell such database-abusers to use BooleanQuery instead.

Actually my idea was to keep PrefixCodedTerms changes out of the game. It stays internal class an it will not be modified at all.
We just need a java.util.stream.Collector (or similar, I'd prefer that as it is a mutable reduction) that produces a PrefixCodedTerms instance from a stream of BytesRef. Then you can feed in any bytesref container (array, collection with or without sorting and deduplication).
Let me hack this together later.
Another problem I see is the following: The current code uses IndexOrDocvaluesQuery.  This is in my opinion the main issue, not the sorting alone.

Another problem I see is the following: The current code uses IndexOrDocvaluesQuery.  This is in my opinion the main issue, not the sorting alone.

I see this as a feature, not a bug. As you can see, left unchecked, "database functionality" can be quite invasive on the codebase. There are all these possibilities for this "join" query:

Terms
DV
Points
Terms + DV
Points + DV

I think it is the correct tradeoff to just have 3 simple queries, one for each underlying index structure, and use IndexOrDocValues to combine them. It keeps this functionality maintainable and in check.

I see this as a feature, not a bug. As you can see, left unchecked, "database functionality" can be quite invasive on the codebase.

The separate query is of course fine, the problem is only that child nodes of the IndexOrDocavlauesQuery may have an expensive ctor (like the one popuplating PrefixCodedTerms).
Basically if all would be in MTQ rewrite this could be handled. If we keep IndexOrDocvaluesQuery we would need to make the soring and PrefixCodedTerms lazy. A good idea would be to make the ctor arguments of IndexOrDocvaluesQuery lazy (producers or functions) and the actual rewrite will create the subqueries. But this would break equals and hashcode, as IndexOrDocavluesQuery relies on the hashcode and equals of the childs.
Because of this MTQ would be much better. The MTQ instance knows everything for equals/hashcode and the rewritten form is lazily created.

If it was done as MTQ, the points+dv case would still need some integration (e.g. IndexOrDocValuesQuery(PointInSetQuery, MTQ(using docvalues)...). So I imagine that case would not get any better, which is sad since if you want to do this stuff, using points is really the way to go: you must make your IDs fixed width or numeric, but they execute much faster than the terms versions. no ordinal lookups in each segment, better cost estimation, etc.

Here is my alternative approach without any crazy addition of collections to PrefixCodedTerms (I just placed the collector there as I needed a common place to be reachable from both queries without adding another utility class): PR #12141
I have not checked performance. I am happy that this shitty instanceof checks are gone. Looks much cleaner to me.

For now I kept the TreeSet in KeywordField, but we can think about that separately. I don't like it too much. Maybe a lazy stream would be best to prevent double sorting. See discussion previously about the downside of IndexOrDocvaluesQuery.

I think we can also close this one, correct? I already closed my proposal: #12141


self in a module with requires_ancestor is not inferred to the ancestor

Input
Note: a link to sorbet.run is not provided, as it does not support requires_ancestor feature.
# typed: true

module M
  extend T::Sig
  extend T::Helpers

  requires_ancestor { A }

  def foo
    bar(self) # !!!: self should be allowed here.
  end
end

class A
  extend T::Sig

  sig { params(arg: A).returns(A) }
  def bar(arg)
    arg
  end
end

Observed output
test.rb:10: Expected A but found M for argument arg https://srb.help/7002
    10 |    bar(self)
                ^^^^
  Expected A for argument arg of method A#bar:
    app/models/1.rb:17:
    17 |  sig { params(arg: A).returns(A) }
                       ^^^
Errors: 1

Expected behavior
No errors! Great job.

similar issue here

This is actually expected, having M.requires_ancestor { A } does not mean that M itself is a subtype of A but that any subtype of M is a subtype of A.
In you case you may be able to workaround this with a cast:
  def foo
    bar(T.cast(self, A))
  end

[sorbet.run]
Note: you can pass ?arg=--enable-experimental-requires-ancestor in the URL to enable the feature.


Add callback setters to TrainableModel

Summary
Add callback getters and setters to the TrainableModel.
Details and comments
This allows resetting the callback in the neural network classifiers and regressors. This is useful for instance to store callback results in different objects.
Not sure if I should add a releasenote on this one, but let me know if you want one!

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

3 of 4 (75.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.01%) to 86.965%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

qiskit_machine_learning/algorithms/trainable_model.py
3
4
75.0%

Totals

Change from base Build<PHONE_NUMBER>:
-0.01%

Covered Lines:
2989

Relevant Lines:
3437

💛  - Coveralls


Correct URL in docs for sensitivity Label or error in the API?

I see that /me/informationprotection/sensitivityLabels is working while /me/security/informationProtection/sensitivityLabels is not working. And the API works on both v1.0 and beta.
Is the docs or API wrong?

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 5ea41611-4cd5-c01e-1acc-29ee0098e8c3
Version Independent ID: 49abf984-3cfe-07cc-5c70-ad22a4402226
Content: Get sensitivityLabel - Microsoft Graph beta
Content Source: api-reference/beta/api/security-sensitivitylabel-get.md
Product: security
Technology: microsoft-graph
GitHub Login: @tommoser
Microsoft Alias: MSGraphDocsVteam

I'm trying to use from documentation the
GET beta/me/security/informationprotection/sensitivitypabels
I'm getting 400 Bad Request
I have the necessary delegated consent "InformationProtectionPolicy.Read" on the access token but I'm not sure what is missing to get this to work or is something on the API
In Graph Explorer is working but when I make a request from my application it doesn't. If I use others API URL it works like
GET beta/me
Need to know if is something on the API used outside the Graph Explorer !?

/me/security/informationProtection/sensitivityLabels is the correct API and should be working. It's available only under /beta.
The other API is used by first-party services only and should be under beta and /v1.0.


Is there a way to not import some users from AD based on some conditions in LdapAttributeHandler?

I want to add additional fields in my user table, and from what I read, I can use the LdapAttributeHandler to add any new fields. But is there a way to drop a record from being imported if it fails some validation?

Unfortunately the only validation you can have is inserting a raw filter into the command:
// In this case, only users that contain an email address and IP phone will be imported.
php artisan adldap:import --filter="(&(mail=*)(ipphone=*))"

You would insert this filter into your config/adldap_auth.php file in the limitation_filter option to have the same effect for users logging in.
Also to clarify adding new fields, you can just add the field into your sync_attributes config option and it will be synchronized onto your model automatically. An LdapAttributeHandler doesn't need to be created for attributes that don't require any logic to parse.

There are before and after methods I've added to the AdldapAuthUserProvider to be able to perform tasks on the discovered LDAP user, but these don't allow you to bail out of the authentication:
https://github.com/Adldap2/Adldap2-Laravel/blob/master/src/AdldapAuthUserProvider.php#L115
https://github.com/Adldap2/Adldap2-Laravel/blob/master/src/AdldapAuthUserProvider.php#L103
I actually may add this type of functionality into the method handleDiscoveredUserWithCredentials(User $user), and if false is returned, bail out of the authentication returning false. What do you think?

Thank you for clarification.
Also, I added the line
   'limitation_filter' => env('ADLDAP_LIMITATION_FILTER', '(mail=*)'),
however, it still brings the users that do not have email. I made the email not null in database, and the insert would fail, but it tries to insert them and pulls from AD lots of users with no email.

Are you using the import command? If so, you need to specify the filter every time. It doesn't pull the filter from your configuration.

The functionality with handleDiscoveredUserWithCredentials would be great, because I could hook into the Eloquent model and pull from database additional data that I need to validate against. For now, all I need is to add a UUID for users that do not have one in my database. But I expect I will need to make additional validations in time, as I expect this project to become large. I will need to look more closely at that method.

I was using the import command. I expected it to use the config filter. I will use it in the command line from now on. Thank you for advice. Is much appreciated.

Woops - reopening. Going to patch this.
The import command will now use the configured limitation filter.
Thanks!


[Training] Issuse while convert model to onnxruntime with awq quantize model

Describe the issue
The error : Valid precision + execution provider combinations are: FP32 CPU, FP32 CUDA, FP16 CUDA, FP16 DML, INT4 CPU, INT4 CUDA, INT4 DML
Extra options: {}
/usr/local/lib/python3.10/dist-packages/transformers/models/auto/configuration_auto.py:961: FutureWarning: The use_auth_token argument is deprecated and will be removed in v5 of Transformers. Please use token instead.
warnings.warn(
GroupQueryAttention (GQA) is used in this model.
Traceback (most recent call last):
File "/content/builder.py", line 1606, in make_model
from quantized_model import QuantModel
ModuleNotFoundError: No module named 'quantized_model'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/content/builder.py", line 2615, in
create_model(args.model_name, args.input, args.output, args.precision, args.execution_provider, args.cache_dir, **extra_options)
File "/content/builder.py", line 2508, in create_model
onnx_model.make_model(input_path)
File "/content/builder.py", line 1608, in make_model
from onnxruntime_genai.models.quantized_model import QuantModel
ModuleNotFoundError: No module named 'onnxruntime_genai.models.quantized_model'
To reproduce
I have  error while running this command code :
!python builder.py -m hiuman/llama-3.1-8B-intruct-awq-quantization -o llama-3.1-8B-intruct-awq-quantization-onnx -p int4 -e cuda -c cache_dir
I use the awq quantized version of llama3.1-8B-Instruct
Urgency
No response
ONNX Runtime Installation
Built from Source
ONNX Runtime Version or Commit ID
1.19.0
PyTorch Version
2.3.1+cu121
Execution Provider
CUDA
Execution Provider Library Version
onnxruntime-genai==0.3.0

Support for building ONNX models from quantized PyTorch models is not currently in the stable package. You can clone this repo and either build ONNX Runtime GenAI from source or run the "from source" command to access this feature until the next stable package is released. We are working through some validation steps and hope to release v0.4.0 of ONNX Runtime GenAI soon.

Closing this issue as it is a duplicate of this one.


Uniqueness

Since validation rules are declared as static, it is not possible to use instance variables to validate models for uniqueness?
When updating a user for example, you want to ignore uniqueness for the user being updated. You can do this by specifying the ID of the model to ignore. Otherwise the validation rules will fail as the user you are updating has the same email address as the user you are updating.
'email' => 'unique:users,email_address,10'

By using a Validator service you can pass an instance of the model into a method and return a validator object for that model which you can check on save.
namespace Models\Services\Validators;

use Models\User;
use Validator;

class UserValidator {

   public static function getValidator(User $user) {
       $rules = [
           'first_name'  => 'required|max:50',
           'last_name'   => 'required|max:50',
           'email'       => "required|max:100|email|unique:users,email{$user->id}"
       ];

       return Validator::make($user->toArray(), $rules);
   }

}

You can't do this with a static variable on a model.
// This will fail if you are updating an existing user.
public static $rules = array(
    'email' => 'required|email|unique,users,email',
);

It seems this should be fixed by buildUniqueExclusionRules()


Add device /dev/kvm

Please give a short reason for introducing this change in the PR description.

Looks great!
Feel free to merge. Once merged follow the release procedure, and/or ask @CommanderK5 to walk you through this process.
The release should be a patch version update.


Fix rejecting handlers of non-derived events

Also, fix the corresponding example

PR is incomplete. I'll upload a fix later.


tpl/os: remove 1mb limit for readFile.

Instead of adding an option that would allow to change the limit, we just remove this limit.
More context here: https://discourse.gohugo.io/t/error-calling-readfile-file-is-too-big/25460.
Will need to update the docs too.

THANK YOU!!

THANK YOU!!

w00t


Conflict with sidekiq

Currently, last version of this gem is 2.2.2 and it has not update new code below
https://github.com/the-teacher/the_comments/blob/master/lib/the_comments.rb#L13
module TheComments
  COMMENTS_COOKIES_TOKEN = 'JustTheCommentsCookies'

  class Engine < Rails::Engine
    config.autoload_paths += Dir["#{config.root}/app/controllers/concerns/**/"]
    config.autoload_paths += Dir["#{config.root}/app/models/concerns/**/"]
  end
end

so it conflict with sidekiq
In order to fix this problem, I have to add this gem directly from git
gem 'the_comments', git: 'git@github.com:the-teacher/the_comments.git', branch: 'master'

Should we have a new version of gem ?
Thanks for help!

ok, I'll do that today

I am waiting you release new version of gem and remove warning message
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
TheComments
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
WARNING!
StateMachine patch for Rails4 will be applied

> private method *around_validation* from StateMachine::Integrations::ActiveModel will be public

https://github.com/pluginaweek/state_machine/issues/295
https://github.com/pluginaweek/state_machine/issues/251
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

https://rubygems.org/gems/the_comments/versions/2.3.1


List potential venues with pro/con - pricing

@DinisCruz to create a central page with this information

Here are the details about the 3 places I've looked at https://github.com/OWASP/owasp-devseccon-summit/blob/master/Logistics/venue

This is now done (we chose CentreParcs)


Make autometrics work easily with http.HandleFunc, and specify 'error' status codes per handler

This very, very likely implies writing a wrapper function to be used as middleware just like the TS version does. It should use the facilities from #47 to inject the traceID and spanID in the context, and wrap the response writer in order to read the status code and automatically mark the handler as successful or not.
This also calls for an addition to  the autometrics:inst directive, so that each function could specify the range of status codes that should be considered an error from the point of view of autometrics. It will default to anything >400 and could optionally take (multiple) --http-code-ok-range 100,499 --http-code-error 404 arguments to override the range that's considered OK (in a "last rule wins" manner)

Since the autometrics wrapping is done by hand for the http.Handler functions for now, and not with a directive comment, there's is no need for now to add the --http-code-ok-range like arguments for now. Instead there is an API that developers will use when calling the middleware to wrap their handlers.


Set-DbaDbCompression - bugs and feature request all-in one

Bug Report
Errors when running command to compress all databases:
Incorrect syntax near 'NO_GAIN'.
General Troubleshooting steps

[x] Verified running latest release of dbatools?
[x] Verified errors are not related to permissions?
[x] Can duplicate in new/fresh PowerShell.exe session?

Version Information

Operating System (Name|Version): Windows 10 / Windows 2012 R2
PowerShell Version: 5.1
SQL Server (Edition|Version): 13.0.4422 STD

Steps to Reproduce

Feature Request
In order of importance

Have ability to choose compression by ROW/PAGE
If MaxRunTime not specified, run as long as you needed
Have switch to just compress all database objects, no matter what expertise have to say =) (in case if `- PercentCompression 0' is not working as CompressAllDatabase)

Problem to solve
Some time we need more brainless things.

Just saw this, will work on the bug fix first then the feature requests.

Question on the cmdlets Test and Set-DbaDbCompression, first off, should Set call Test? They both run pretty much the same T-SQL query to get the results, would also be useful to run something like this so you could review suggestions, maybe manually remove some if wanted and then pass into Set.
Thoughts welcome, trying to use this in my environment and I feel like it needs some love and looking for some guidance.
$compressionSuggestion = Test-DbaDbCompression -Sqlinstance Server1
Set-DbaDBCompression -InputObject $compressionSuggestion

#FEATURE REQUEST
Set-DbaDbCompression -SqlInstance Server1 -Database DB1 -Compression Page
And all database will be compressed with DATA_COMPRESSION = PAGE, very useful for DEV/TEST environments where we short on storage space.

@Zhunya I'll make sure to add that in too, agreed that would be useful.

Closing


KeyError: '000000F'

Getting the following error when running finisherSC.py
1: PREPARING DATA
2,3: RUNNING mummer AND CREATING CLUSTERS
# reading input file "/Volumes/HD/lt/Desktop/assembly/self.ntref" of length 1326462
# construct suffix tree for sequence of length 1326462
# (maximum reference length is 536870908)
# (maximum query length is<PHONE_NUMBER>)
# process 13264 characters per dot
#....................................................................................................
# CONSTRUCTIONTIME /usr/src/app/finishingTool/MUMmer3.23/mummer /Volumes/HD/lt/Desktop/assembly/self.ntref 0.33
# reading input file "/Volumes/HD/lt/Desktop/assembly/contigs.fasta" of length 1326461
# matching query-file "/Volumes/HD/lt/Desktop/assembly/contigs.fasta"
# against subject-file "/Volumes/HD/lt/Desktop/assembly/self.ntref"
# COMPLETETIME /usr/src/app/finishingTool/MUMmer3.23/mummer /Volumes/HD/lt/Desktop/assembly/self.ntref 1.02
# SPACE /usr/src/app/finishingTool/MUMmer3.23/mummer /Volumes/HD/lt/Desktop/assembly/self.ntref 2.55
4: FINISHING DATA
Traceback (most recent call last):
  File "finishingTool/finisherSC.py", line 76, in <module>
    mainFlow(newFolderName, newMummerLink, args['pickup'], args['mapcontigs'])
  File "finishingTool/finisherSC.py", line 19, in mainFlow
    nonRedundantResolver.removeEmbedded(folderName , mummerLink)
  File "/usr/src/app/finishingTool/nonRedundantResolver.py", line 31, in removeEmbedded
    l1, l2 = lenDic[name1], lenDic[name2]
KeyError: '000000F'

I installed from GitHub and ran make clean and make all. make check says all is ok. Any thoughts?

Would you mind trying to run the test set provided in the sandbox example first?
As for the key error, is "000000F" a contig name? Also, can you run 'sed' in command line properly, it is needed for finisherSC to run properly.   But you can shoot me an email with the files that you are working on and I can take a closer look.

The example dataset runs fine. Yes, 000000F is a config name that produced from running FALCON.
>000000F 000000380:E~000000043:B~000004270:E~000004270:E ctg_linear<PHONE_NUMBER>32 is the full config name. I checked sed and it is version 4.2.2 and seems to run ok.

The weird thing is if I just modify that first FASTA header to >000000F, but leave all the rest in the same format it completes just fine.

I see. It actually depends on how MUMmer handle header's name as well. If you don't mind, how about changing the contig name to be something like Seg1, Seg2, ...etc ?

I can patch the file. Thanks.

Hey, @lennythomas,  you may also want to try this to do the name patching
    perl -pe 's/>[^\$]*$/">Seg" . ++$n ."\n"/ge' oldFileName > newFileName

Hi, I've been having this same issue the past few days. I have tried everything suggested above and I am still having the same issue. Any suggestions?

Same, after name patching, still got same error.


Update share-to-teams.md (#707)

Docs Build status updates of commit 802d0cd:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

msteams-platform/share-to-teams.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Implement Attributes in rbx_dom_lua

This builds on top of some of the work in #218.

This code appears to be functional in Rojo.


WIP: Adding compatibility for API supporting HPA

draft for changes required for https://github.com/kubeflow/common/pull/206

Closing it as changes are added in #1724


Replicate DevOps changes in punters-client-1.0.0a2

(closes #11)

Coverage remained the same at 100.0% when pulling 9dcd43a3ddb3925379bb3e8ecff23842e1190fb2 on versions into f42459019412a927a4a1a85b1eda441b1598a0ec on master.

Coverage remained the same at 100.0% when pulling 3a64f84fb03968cdd01aea190440bc57a9bdd886 on versions into f42459019412a927a4a1a85b1eda441b1598a0ec on master.


add_comment function added

Press c to add a comment on the submission if a header is selected, or reply to a comment if the comment is selected.
Due to curses bug, textbox can only work on overwrite mode (insert mode not working).
p.s. Ignore the last pull request. I combined all the commits.

Thanks, I'll look over this later today

What's the purpose of show_cursor=False? It looks like on my system, neither that or insert_mode=True actually do anything. I don't get the recursion bug either. I wonder if we are pointing to  different curses libraries.

Here is what I have:
Python 2.7.8 (default, Sep 29 2014, 18:17:43)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.51)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import curses
>>> curses.version
'2.2'

On my environment, curses.curs_set(1) is needed to show the block cursor. curs_set(2) does not show anything.
The flag insert_mode=True tries to insert a letter in place of the cursor and push back every single letter behind it by one. However, recursion is used for this process, and if the textpad area is too large, Python calls too many recursions and the following error occurs:
RuntimeError: maximum recursion depth exceeded while calling a Python object

It is described here

Here's what I have for reference:
Python 2.7.5 (default, Oct 13 2014, 12:50:39)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import curses
>>> curses.version
'2.2'

I think we should use the following settings to maintain stability across the largest number of systems:

curses.curs_set(1). On my environment curs_set(1) and curs_set(2) look the same. I am fine sticking with 1 as a rule of thumb.
insert_mode=False. This would be really nice to have, but since it's unstable ATM I think we have to leave it out. Attempting to patch the curses module may break future python releases. The other option would be, as you said, to re-implement the textpad. However I think we would run into the same problems trying to maintain anything curses related across different os's and curses versions.
strip_spaces=False. This can be done just as well manually via string.rstrip(' ')

I merged this and added some additional changes based on my comments above 9ecf7e1. If you get a chance, please try this out on the master branch and see if everything still works for you! Feel free to open up a new issue if you find anything.
Thanks again for the pull request.

Looks good. Sorry I didn't have time to work on it past few days.


update version number and dependencies numbers

Description
update version numbers for skylab v<IP_ADDRESS>.0-rc1
the following upstream still need to be updated when they are updated.

[ ] saber
[ ] ioda
[x] fms ? (probably not)
[x] mom6 ? (maybe)

... didn't feel like waiting for ioda/saber numbers to be bumped up


Packaging Request: Tuxedo Control Center

Project description
Tuxedo is a German laptop manufacturer that provides Linux-friendly laptops. Their system control is done via an app called "Tuxedo Control Center" (TCC). This open source app provides fan control settings among other things. Without this app, the Tuxedo laptops default to very noisy fan control settings. It lives on Github.
Besides the source, Tuxedo provides binary packages for Ubuntu and OpenSUSE.
TCC consists of:

a set of systemd services implemented in Javascript,
an Electron GUI application to control everything.

Metadata

homepage URL: https://www.tuxedocomputers.com/en/TUXEDO-Control-Center.tuxedo
source URL: https://github.com/tuxedocomputers/tuxedo-control-center
license: GPL3
platforms: Linux

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/packaging-the-tuxedo-control-center-electron/14326/2

i'll take a shot at this

builds and launches but complains about
/bin/sh: prime-select: command not found
/bin/sh: prime-supported: command not found

which are provided by a fedora only package
will also need a module as tccd service is needed
{ lib
, stdenv
, makeWrapper
, fetchurl
, rpmextract
, autoPatchelfHook
, alsa-lib
, cups
, gdk-pixbuf
, glib
, gtk3
, libnotify
, libuuid
, libX11
, libXScrnSaver
, libXcomposite
, libXcursor
, libXdamage
, libXext
, libXfixes
, libXi
, libXrandr
, libXrender
, libXtst
, libxcb
, libxshmfence
, mesa
, nspr
, nss
, pango
, systemd
, libappindicator-gtk3
, libdbusmenu

}:

stdenv.mkDerivation rec {
  pname = "tuxedo-control-center";
  version = "1.0.14";

  src = fetchurl {
    url = "https://rpm.tuxedocomputers.com/opensuse/15.2/x86_64/tuxedo-control-center_${version}.rpm";
    sha256 = "1fhcxf6jdb2b79pkyjxz6l78m8gbzlv0swncys6r17i08y2vv8d1";
  };

  nativeBuildInputs = [
    rpmextract
    makeWrapper
    alsa-lib
    autoPatchelfHook
    cups
    libXdamage
    libX11
    libXScrnSaver
    libXtst
    libxshmfence
    mesa
    nss
    libXrender
    gdk-pixbuf
    gtk3
  ];

  libPath = lib.makeLibraryPath [
    alsa-lib
    gdk-pixbuf
    glib
    gtk3
    libnotify
    libX11
    libXcomposite
    libuuid
    libXcursor
    libXdamage
    libXext
    libXfixes
    libXi
    libXrandr
    libXrender
    libXtst
    nspr
    nss
    libxcb
    pango
    systemd
    libXScrnSaver
    libappindicator-gtk3
    libdbusmenu
  ];

  unpackPhase = ''
    mkdir -p $out/bin
    cd $out
    rpmextract $src
  '';

  installPhase = ''
    runHook preInstall

    wrapProgram $out/opt/${pname}/${pname} \
        --prefix LD_LIBRARY_PATH : ${libPath}:$out/opt/${pname}

    ln -s $out/opt/${pname}/${pname} $out/bin/

    runHook postInstall
  '';

  meta = with lib; {
    description = "A tool to help you control performance, energy, fan and comfort settings on TUXEDO laptops.";
    homepage = "github.com/tuxedocomputers/tuxedo-control-center";
    license = licenses.gpl3Only;
    maintainers = with maintainers; [ ];
  };
}

Awesome! I think this is an excellent first step. If I have some time, I'll try to get the systemd services running, but I don't mind if anyone beats me to it. :)

@arturrin patchelf seems to cause issues with tccd. I'm hitting this issue: https://github.com/vercel/pkg/issues/321. I've tried to get around patching the binary by adding all dependencies to the LD_LIBRARY_PATH of the wrapper, but this results in weird glibc errors. So I currently assume to package tccd we need a hack like this one.
I wonder this hack can be avoided by building everything from source.

Btw, looking at the code, the prime-select errors are somehwat harmless and probably just prevent nVidia PRIME configuration from working.

Not sure if it's helpful but I had this packaged privately a while ago and working for my clevo-based laptop:

module: https://github.com/Vodurden/devbox/blob/f06f565286e069ec5c58c222c97e37f0f7c5b551/nix/modules/nixos/tuxedo-control-center.nix
tuxedo-cc-wmi: https://github.com/Vodurden/devbox/blob/f06f565286e069ec5c58c222c97e37f0f7c5b551/nix/pkgs/tuxedo-cc-wmi/default.nix
tuxedo-control-center: https://github.com/Vodurden/devbox/tree/master/nix/pkgs/tuxedo-control-center

I'm not willing to submit it upstream since I no longer use the module and I think tuxedo control center has been re-organised since I wrote the package, but I figure this might help someone in building a new derivation :)

@Vodurden Oh wow. Very nice! This is very helpful. I'm going to give this a go with the new TCC version. I'm also happy to maintain this once its in nixpkgs.

I started working on this in  https://github.com/blitz/nixpkgs/tree/tuxedo-control-center
Still to go:

updating TCC to the most recent version,
integrating the module from @Vodurden

TCC is updated to 1.0.14 in my branch (latest version). This leaves the module.

I've added the module. So I think we are "code complete" here. I'll have access to a Tuxedo laptop in 2 weeks to test. If someone wants to check it out in the meantime, rebuild your NixOS with https://github.com/blitz/nixpkgs/tree/tuxedo-control-center and enable hardware.tuxedo-control-center.enable = true;

I'm glad to see the module/derivation was helpful, thanks for taking it on and giving it a new life! :)

I'd like to thank you all for working on this and I hope it gets merged.
I have a new Clevo NH55EJQ laptop (from DreamMachines.io) running NixOS 21.05 (soon to be 21.11), and this tuxedo-control-center package (currently installed following https://github.com/blitz/tuxedo-nixos#usage) is the only thing I've found that allows control over my laptop's performance profile.  Without it, the CPU GHz, boost, and fan speed were below expected, but with it I'm able to change the profile to achieve maximum performance (or choose a quiet profile).
Tuxedo laptops are (usually? always?) Clevo models (I heard), and I learned from the Arch Linux wiki (I think it was) that the Tuxedo Control Center often works for other Clevos.  There is an official Control Center app for Windows for Clevos that is mentioned in my laptop's User's Manual.  It's great to have something like it for Linux.
I can confirm that, without prime-select and prime-supported, this tuxedo-control-center package still works for CPU and fan control.  My laptop does have an Nvidia dGPU but I haven't gotten into enabling it yet.
I'd be glad to help further test this package if needed (within my limited familiarity with Nix and Electron).

@DerickEddington Thanks for the warm words. :) To my knowledge all Tuxedo laptops are Clevo laptops. I'm going to make another push to get this into nixpkgs soon. The main obstacle right now are:

finding the right way to package the TCC electron app in a way that placates the nixpkgs maintainers,
https://github.com/tuxedocomputers/tuxedo-control-center/issues/148

The first point I can probably defeat on my own. For the second, I would need some help from someone who actually knows the JS/Electron ecosystem.

Upstream said they will bump Electron soon. 🎉

Any News on That? Any More help Required?

@marvin (or anyone else interested in moving this forward :) )

Any News on That? Any More help Required?

Upstream is very slow to update Electron. They basically update to versions that are already deprecated. If you have the skills to help with that, head over to: https://github.com/tuxedocomputers/tuxedo-control-center/issues/148
Updating the Nix expressions to TCC 1.1.4 has also been a challenge and I didn't have time to figure out the issues. 1.1.4 hits some node2nix issues, that i don't understand yet. If you want to take a shot, the update instructions are here: https://github.com/blitz/tuxedo-nixos/tree/master/nix/tuxedo-control-center

Updating the Nix expressions to TCC 1.1.4 has also been a challenge and I didn't have time to figure out the issues. 1.1.4 hits some node2nix issues, that i don't understand yet.

I got version 1.2.2 (latest as of writing) working locally (well, and in one big ugly commit). I am currently working on splitting the update into the update itself and various small fixes and tweaks.
I hope to be able to open a PR in the blitz/tuxedo-nixos repository with the update in the following days :)


Consistent test failures when run against CA signed cert configured server

Summary
We have two tests that do no pass (and should not pass) when running against a Conjur server machine/LB that has been configured with a CA signed certificate.
test_https_cli_fails_if_cert_is_bad and test_https_cli_fails_if_cert_is_not_provided. The reason they should not pass is because these two tests check the cert_file field of the conjurrc and with the CA signed cert flow, we do not use that field to verify certificates. Instead we verify the certificate provided by the server on each request against known CA bundles on the machine. In other words, we don't use the cert_file field contents
FYI @eladkug
Steps to Reproduce
Steps to reproduce the behavior:

Pack the cli and the test runner pyinstaller -F test/util/test_runners/integrations_tests_runner.py and pyinstaller -D ./pkg_bin/conjur on macOS

Run
./dist/integrations_tests_runner
--identifier test_with_process
--urlhttps://sigallb.aim-dev.conjur.net/
--account cucumber
--login admin
--password blah
--files-folder test
--cli-to-test ./dist/conjur/conjur

See first two tests failing

Expected Results
Either fix the flow or add a filter on the tests that should be run
Reproducible

[x] Always
[ ] Sometimes
[ ] Non-Reproducible

In the interest of maintaining compatibility between the Ruby and Python CLI's handling of .conjurrc, conjur-api-python#37 allows the SSL verification to work with the system trust store regardless of the cert_file value. The cert_file property can still be used via the --self-signed flag to trust a self-signed cert, and via the --ca-cert flag to trust a root CA outside of the system trust store. But if the pem file it points to isn't a root CA or self-signed cert, the connection can still succeed if the system trust store contains the associated CA.
We should re-evaluate the associated tests in the Conjur CLI to verify this behavior against a CA-signed certificate.


Investigate refactoring of Linux installer gradle build process to aid automation

Part of #1005
Investigate above.

Gradle builds working with new templating process.


Feature Request : possibility to run task from folder in Explorer panel

It is very helpful to add the possibility to run one of the configured task on a specific folder in the project, by right clicking on the folder in explorer and chose the task, the folder path then will be used as the option  "cwd". this give more flexibility for big projects with Makefiles inside each folder.

I think having https://github.com/microsoft/vscode/issues/71461 would be a neat solution for having lots of makefiles. You could have one task that prompts for a folder and runs the make.


tests fail without fs_peptide data

Many of the tests require fs_peptide/fs-peptide.pdb
but it looks like from the .gitignore that this directory is excluded from the repo
Would it be possible to include the test data in the repo, or alternatively host the test data remotely and give a link to it, so the tests can be run?

That's bad! For now, you can find that pdb here, along with a handful of other datasets in msmb_data.
@mdtraj/developers: maybe we should look into running the tests on data from msmb_data?

Hi @momeara
The test data can be downloaded here https://figshare.com/articles/Fs_MD_Trajectories/1030363
It's too big (probably) to include it in the repo, but there should definitely be a link and documentation
@cxhernandez note this repo is tftraj not mdtraj

LOL my bad

cheers!


Attach amplify app's analytics to existing pinpoint project.

Note: If your feature-request is regarding the AWS Amplify Console service, please log it in the
official AWS Amplify Console forum

First Created a pinpoint project in pinpoint console.
Then I want to add analytics features.
So I create an amplify app by CLI like follows:
amplify init
amplify add analytics
amplify push
The amplify app's analytics will create and link to a new pinpoint project.
Is there a way that could link amplify app's analytics to existing pinpoint project?
Is your feature request related to a problem? Please describe.
My partner created a pin point project first and already did lots of setting tests on the project.
So we want to import analytics to the existing pinpoint project.
Describe the solution you'd like
1.Command like:
amplify add analytics --projectId
2.Confingure awsconfiguration.json and run
amplify  update analytics
amplify push
"PinpointAnalytics": {
"Default": {
"AppId": "your-pinpoint-project-id",
"Region": "us-xxxx-x"
}
},

Simply configure the analytics in amplify console.

Describe alternatives you've considered
Additional context
Add any other context or screenshots about the feature request here.

Amplify CLI does not support adding an existing analytics project. But you can use the existing Pinpoint resources as described in our docsite

Amplify CLI does not support adding an existing analytics project. But you can use the existing Pinpoint resources as described in our docsite

Thanks for your reply. The doc link shows a way to do this with javascript.
I want to find how to do the manual setup for iOS or android apps.
Are there some docs about the manual setup for iOS or android ?

@toka756 Are you using the AWS Mobile Android SDK or the Amplify SDK for your iOS/Android app?
We have documentation for manually adding the analytics config out here - https://aws-amplify.github.io/docs/android/analytics#flush-events
Moving this over to the android github repo for help.


Make / only route to start page on prod

We change route so it doesn't redirect to the live environment in the
dev environment.

@SteveMarshall Tested on pre-prod


New package: ComplexNoise v0.1.0

Registering package: ComplexNoise
Repository: https://github.com/brian-sinquin/ComplexNoise.jl
Created by: @brian-sinquin
Version: v0.1.0
Commit: 16c13d676489c18fca46911ce92d6a8983df24f6
Reviewed by: @brian-sinquin
Reference: https://github.com/brian-sinquin/ComplexNoise.jl/commit/16c13d676489c18fca46911ce92d6a8983df24f6#commitcomment-142938941

Hi! You should probably write some code and documentation before registering!

Empty packages are not suitable for registration, see https://github.com/JuliaRegistries/General?tab=readme-ov-file#should-i-register-my-package-now.
You're welcome to register again when you have some substance in the package.

Sorry about that and thank you for your time
I'll fill some code soon


Implications of Secure Retrieve Trial Implementation

When asked to propose changes on MHDS due to the update of the Secure Retrieve Trial Implementation, I faced some open questions to be discussed in the ITI Technical Committee.
Comparing the options in MHDS with the options in its XDS.b counterpart, I realized that
XDS.b mainly defines actor options for the core document management, e.g. Asynchronous Web Services
Exchange, On-Demand Documents, PIX Feed for patient identity cross referencing. An exception is the
Basic Privacy Enforcement option which adds functionality for authorization, without going into details,
particularly an option for using XUA to convey the user roles and attributes for the role/attribute
based authorization decisions is not defined in XDS.b. This is fine, since the the ITI technical
framework is modular and any actor of the profile can be grouped with any other actor of the
framework, if one requires a specific functionality and the corresponding transactions in an
application.
MHDS on the other side describes the details related to the consent manager and authorization option,
i.e., describes in detail how access decision shall be made by the authorization server actor based in
the consent and the requirement to enforce the decision in the resource server actor, but does
not distinguish between the consent given to authorize the access to resources for the (mobile)
application and the consent to authorize access to resources for the user trying to access using
the application (mobile).
While the first is managed by the IUA authorization server, the latter is not intended by IUA. IUA is
based on OAuth which also intends to handle the authorization of applications (or clients) to act
on behalf of the user, but not the authorization of the user to access the resources
(see https://profiles.ihe.net/ITI/IUA/index.html#34-internet-user-authorization-iua-profile
and https://datatracker.ietf.org/doc/html/draft-ietf-oauth-v2-1-07#name-introduction-4).
But like in XUA, in IUA we use the OAuth token extensions to transport some additional attributes
(e.g., role, purpose uf use, etc.), which shall be used by the resource server to make access
decisions based on the resource owner consent. Here Secure Retrieve comes into play: Instead of
implementing the decision making in all resource servers of your infrastructure, you can delegate it
to the Authorization Decision Manager actor defined in the Secure Retrieve Trial Implementation and
use an Authorization Decisions Query [ITI-79] transaction to retrieve the access decisions.
Secure Retrieve is more suitable for authorization decisions based on complex rules defined in the
consent. In simple cases the use of Secure Retrieve may add to much complexity and the access decision
can be made be the resource server or even by extending the IUA authorization server. Such cases are
currently discussed in the FHIR community in the context of Smart on FHIR. This approach is handy for
simple situations but will quickly become problematic for more complex authorization rules.
So back to the main question: What is the implication of Secure Retrieve for MHDS?
Answer 1: None, if the current description focusses on simple situations only and alle access
decisions can be made by the IUA authorization server (maybe be extending the scopes defined in IUA).
Answer 2: Add an option for complex situations, in which the authorization rules are complex enough
to justify the use of Secure Retrieve. Using Secure Retrieve the MHDS Document Registry optional
grouping with the IUA resource server actor shall be replaced with the Authorization Decisions
Verifier actor defined in the Secure Retrieve Trial Implementation.
Answer 3: Remove the Consent Manager Option and the Authorization Option. The ITI technical
framework is modular and any actor of the MHDS can be grouped with any other actor of the
framework, if one needs a specific functionality and the corresponding transactions in your
application. This would simplify the MHDS IG and increase it's readability. We can add a note
in the Security Considerations pointing to the possible groupings to manage authorization.

The impact of PCF and SeR are not clear.


Manage event subscriptions and publications in Fabric Driver in source mode

Here is the interface for event lookup:
async function lookupEventSubscriptions(eventMatcher: eventsPb.EventMatcher): Promise<Array<Query>> {
}

Closing issue as is subscription and publication is handled on driver in source mode.


the pointInFreehand Error

Prerequisites

[4.9.0] Which version are you using? (Is it latest?)
[Yes] Are you reporting to the correct repository?
[No] Did you search existing issues? (Were any related?)

Description
When I use the pointInFreehand to determine whether a point is inside a polygon, I find an error.
A ponit is inside a polygon but the pointInFreehand return false
Steps to Reproduce the issue

Modifying test cases in the file FreehandMouseTool.test

describe('pointInFreehand.js', function() {
  // Simple square
  const dataHandles = [
    new FreehandHandleData({
      x: 0.0,
      y: 0.0,
    }),
    new FreehandHandleData({
      x: 0.0,
      y: 1.0,
    }),
    new FreehandHandleData({
      x: 1.0,
      y: 0.0,
    }),
  ];

  it('should return true if the point is inside the polygon', function() {
    const point = {
      x: 0.1,
      y: 0.8,
    };
    const isInside = pointInFreehand(dataHandles, point);

    expect(isInside).toBeTruthy();
  });
})

npm run test
I see error

:+1: yep，same problem.
there are some problems with  util/pointInFreehand.

lineSegmentAtPoint(Returns the y value of the line segment at the x value of the point.) y value should be lp1.y + dydx * (point.x - lp1.x)
isLineRightOfPoint(Returns true if the line segment is to the right of the point.) need to take gradient.

Commit solves. Can submit a PR for this ? :wink:  @zzfzfz  @dannyrb

@DStar2013, that would be wonderful ^_^
CC: @JamesAPetts

I've just stumbled across this myself! A PR would be awesome and I'd be happy to review it.


Update to v18

Also:

HttpClientModule is deprecated in Angular 18, was updated to provideHttpClient.

@dkhrunov please check it.


Window + FrameView - Can the codebase be simplified by having them share code?

Why does Window have an embedded ContentView.
It seems like un-needed extra complexity.
Both are inherited from View, so both have the same capabilities.
The lifetime of contentView is the same as the Window.
Same question for FrameView.
What am I missing?
And, while I'm at it: What is the difference between Window and FrameView? They do exactly the same thing, as far as I can tell. And they look the same visibly. Why do both need to exist?

As you have a lot of knowledge of WinForms you know that there are controls that group and serve as containers for others so that when you move or reposition a container all their children will automatically reposition themselves all at once without having to reposition them individually each. Of course they have to be derived from View but they don't have to be Toplevel. The FrameView is not derived from Toplevel. The ScrollView have that too because it needed to have a container to accommodate the childrens.

What is questionable is whether ContentView should be dependent only on Window or whether it should be independent like FrameView. My opinion is that it could be, although there are containers that are only dependent on their hosts because only these are the ones who know how to deal with it, such as ScrollView.

I think, based on digging deeper + your explanation, the simple summary is:

Window and FrameView both use an embedded ContentView in order to have a visible frame and present child controls with a 'client area' that subtracts the frame. In other words, the border of the Window/FrameView is the "non-client area" and the ContentView is the "client area".
Window and FrameView operate identically. The only difference is Window derives from Toplevel and thus is a Toplevel view, and FrameView is not a Toplevel.
There are two implementations of ContentView but both do nothing (they are just subclasses of View.

Note Window and FrameView actually declare their private contentView field as a View:
	public class Window : Toplevel, IEnumerable {
		public View contentView;

I ran an experiment where I modified Window to JUST USE View (deleting the defn of ContentView and it all works fine.
In other-words, ContentView is completely un-needed in both Window and FrameView.
I believe the code and design of Terminal.Gui can be simplified by doing this.
I also am now wondering if things could be simplified further by hanging Toplevel to be an interface. By doing that Window and FrameView could share their code.
	public class FrameView {
...

	public class Window : FrameView, IToplevel, IEnumerable {
...

Right now these two classes have duplicated functionality, but use slightly different code to do the same thing. E.g. Here's the two implementations of `Redraw`:

```csharp
		///<inheritdoc cref="Redraw"/>
		public override void Redraw (Rect bounds)
		{
			Application.CurrentView = this;

			if (!NeedDisplay.IsEmpty) {
				DrawFrameWindow ();
			}
			contentView.Redraw (contentView.Bounds);
			ClearNeedsDisplay ();
			DrawFrameWindow (false);

			void DrawFrameWindow (bool fill = true)
			{
				Driver.SetAttribute (ColorScheme.Normal);
				DrawFrame (fill);
				if (HasFocus)
					Driver.SetAttribute (ColorScheme.HotNormal);
				var width = Frame.Width - (padding + 2) * 2;
				if (Title != null && width > 4) {
					Move (1 + padding, padding);
					Driver.AddRune (' ');
					var str = Title.Length >= width ? Title [0, width - 2] : Title;
					Driver.AddStr (str);
					Driver.AddRune (' ');
				}
				Driver.SetAttribute (ColorScheme.Normal);
			}
		}

		///<inheritdoc cref="Redraw(Rect)"/>
		public override void Redraw (Rect bounds)
		{
			if (!NeedDisplay.IsEmpty) {
				Driver.SetAttribute (ColorScheme.Normal);
				DrawFrame ();
				if (HasFocus)
					Driver.SetAttribute (ColorScheme.Normal);
				var width = Frame.Width;
				if (Title != null && width > 4) {
					Move (1, 0);
					Driver.AddRune (' ');
					var str = Title.Length > width ? Title [0, width - 4] : Title;
					Driver.AddStr (str);
					Driver.AddRune (' ');
				}
				Driver.SetAttribute (ColorScheme.Normal);
			}
			contentView.Redraw (contentView.Bounds);
			ClearNeedsDisplay ();
		}

The logic at the top of Window's version could easily be wrapped with a type check.
I don't think this is urgent or critical by any means, but the best programmers remove as much code as they add ;-)

But imagine that in a window you have several containers and each of them has several controls. If you put all these controls in Window only, for example, if you need to move all of these controls to another place, you will have to reposition each one individually, whereas if there are containers, just move the container to another place and it will automatically reposition all of its controls. So, in my opinion, I think it would be a mistake to end the containers once and for all, or else to create a Container View that in turn is inserted in the Window. In this case, that's fine.

I'm not suggesting getting rid of the fact that Window and FrameView have an embedded View that is a container. I'm suggesting we simply don't have the additional layer in the class heirarchy (just have Window and FrameView use View directly as their embedded container.
They do this already, actually. Look at the code. You'll see ContainerView has NO functionality they use; they literally create a ContainerView and assign it to a field of type View.

Ok we can get rid the ContainerView to a new file class  ContainerView.cs and then acttached it to the field in the Window and FrameView.

Ok we can get rid the ContainerView to a new file class ContainerView.cs and then acttached it to the field in the Window and FrameView.

Why not just use View for that? Why have another class derived from View that does nothing?

Ok, I agree. A public View ContentView property, instead of an inline class. Good.

The PoC I show off in #547 proves (to me anyway) that it's possible to get rid of the ContainerView classes in both Window and FrameView.  In fact, it proves that FrameView can be deleted completely.

This shouldn't have been closed as they still use independent code to do the same thing, resulting in varying behavior/bugs. I addressed part of this in #522 by unifying the DrawFrame code, but not other logic.
Another example: These two boxes should look identical:

For me and most important difference between Window and FrameView is that this last is only a group box to frame views inside but without could be dragged, unless they are worked on a IDE, acting as container and not to be as toplevel. Window on the contrary, is a toplevel and is draggabled. Said that, I think it's better to maintain FrameView like it is.

I still hate that Window and FrameView are basically the same but with different code. But it's time to move on. Closing.


Outdated protobuf

Hello. You need to update the reference to protobuf, because currently selected version trying to download gmock files from https://googlemock.googlecode.com/files/gmock-1.7.0.zip, but it's an invalid link now.

This is fixed in the latest commit. We're still at Protobuf 3.2.0rc2, though. They released shortly before.


Add move assignment method to Optional<>

Needed in #20638.
BTW, actually, absl::Optional<> doesn't have any set() method.

You are right. This needs to be fixed

The test failure seems unrelated. Filed in https://github.com/grpc/grpc/issues/21376.


split parameters.jl into smaller files

Resolves #44. Once we've agreed with the structure proposed in this PR, I'll also update the test/ structure to match.

@willtebbutt would you be happy to accept this in principle? then I'll go through and sort out the merge conflict....

Sorry for the slow response @st-- . I like this PR, would be happy to merge it once merge conflicts have been resolved.

@willtebbutt done, ready for review!


Add jq for command line JSON parsing

@juharris @CoatedMoose I will need this deployed too

Moving to that init script instead


i wonder how to creat a task and a robot of my own , i dont even know what the numbers mean in the examples

i wonder how to creat a task and a robot of my own , i dont even know what the numbers means in the example

The numbers are indices into the lists of rules that can be applied at each step. This list of rules changes depending on the previous rules applied, and is very difficult to interpret. They're not really meant to be edited by hand.
It might be easier to run the design search to get more robots? This is how we got the examples in the paper.


debounce-handler: deal with component unmount

Hi there!
It's more of a question here. Let's say we have a callback with really big delay, in that case the callback might be invoked after the wrapped component had already been unmounted. Should debounce-handler do something with that? e.g. it might prevent debounced function from firing after component unmount, or force to run it right before component unmount, to ensure the last call done while component is still alive.

I just ran into this issue - has it been resolved?
This library made it easy to debounce with recompose - thanks!


⚠ Bofrost.at has degraded performance

In af0cef5, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 2504 ms

Resolved: Bofrost.at performance has improved in 6bfe245 after 7 minutes.


Consider data download in CSV format

Allows for non-sighted users to inspect the data.

This would be particularly helpful with data features, to look at your 10 samples you'd just be navigating a table with descriptive names and a number per sample.
If it makes it easier, you could split into 2 files per action, with 1 for x,y,z and one for features. Bundled into 1 zip download.


issue component Weather Buienradar

The weather for today is rigt. The forecast is broken.
I have added 4 different locations. And the daily weather is different and the forecast is the same at each location. Between the locations is +/- 2500 km difference.
This is set in weather.yaml

platform: buienradar
name: 'Doorn'
platform: buienradar
name: 'Moscow'
latitude: 55.74
longitude: 37.62
platform: buienradar
name: 'Pomorie'
latitude: 42.57
longitude: 27.62
platform: buienradar

Greetings justin

/move home-assistant/home-assistant


Support for different units in width/height

Hello, do you have any plans to include support for custom units in the width/height or even positions of actors?
I'm implementing a game with a pixel-perfect UI viewport, with actors' sizes defined in DPs (like Android) and this would be pretty useful for me and maybe for others.

No, I don't actively implement new features in LML if their scope would require too much work, sorry. If anything, I currently focus on KTX.
Numbers you use in your LML files translate directly into your Stage's Viewport units - these are not pixels, unless the viewport treats it as so. You might want to play around with Viewport implementations and see if anything fits your needs.

Actually right after opening this issue I figured a way to do this without changing the lib's code. Instead of placing DPs in sizes and positions it should be easier to make the viewport size in DPs by calculating this on resize(). Anyway, thank you for the libs.

That's great. Anyway, if you have any problems with the library (or some small feature requests), don't hesitate to create new issues.


NEqProblem fails with BFGS and LBFGS

https://github.com/JuliaNLSolvers/NLSolvers.jl/blob/486e562e32322c9443617a935b4ef8a5ec387f59/src/nlsolve/linesearch/newton.jl#L25-L29 asks the scheme for a ´reset_age´ field, but there isnt one. with that, only newton works on NEqProblems. maybe using a function instead of a field access could solve the problem?

I see what you're getting at, but I think everything here should just be annotated Newton. Then, another dispatch can wrap the objective in a meritobjective type and minimize it.


Remove version: Wagnardsoft.DisplayDriverUninstaller version <IP_ADDRESS>

[Automated] It returns code over 400 in all urls
Microsoft Reviewers: Open in CodeFlow

URL: https://www.wagnardsoft.com/DDU/download/DDU v<IP_ADDRESS>_setup.exe
OK
(Automated message - build 911)

Close with reason: Package still available. Closing as requested in other PRs.;


Jetcaster UI updates

Add to Jetcaster: shared elements, swipetodismiss, lazy list animations and removal of episode from queue
https://github.com/user-attachments/assets/95d6c54a-1f62-4bc9-8903-920296dd212b

With PR #1486 merged into the main branch, I have worked on updating the logic in this PR to use only Grid. Would it be alright to contribute these changes to the current branch?

I have modified it so that the AnimationScope is not applied when the device is unfolded and the detail screen is open.
I fixed an issue where SharedElementTransition was not applied in the library list.

If you are working on this separately, feel free to skip this comment.
https://github.com/user-attachments/assets/a2994e92-8fcd-47fb-965b-21c01c97272d

With PR #1486 merged into the main branch, I have worked on updating the logic in this PR to use only Grid. Would it be alright to contribute these changes to the current branch?

I have modified it so that the AnimationScope is not applied when the device is unfolded and the detail screen is open.
I fixed an issue where SharedElementTransition was not applied in the library list.

If you are working on this separately, feel free to skip this comment.
fold.mp4

Hiya @yongsuk44  - thanks for your contributions! Just to clarify - have you already made these changes locally, but just haven't committed to this PR? If so, please let me know, as I was planning to do this today :) just so we don't duplicate the work. If you've already done the work, please feel free to commit to this branch. Thanks again!

@simona-anomis
I've created a separate PR #1492 to merge into the 'jetcaster-ui-update' branch.
Could you please take a look? 🙂


How to get Password?

Sorry I'm new to this. I want to try the docker image but I cannot login to Jupyter Notebook. How can I get the password?

the password is mentioned in the README: dl_course


[BUG] plugin-handle list bug

plugin-handle has a problem submitting a re-fetch list request URL after editing.
eg:
http://localhost:9095/plugin-handle?pluginId=&currentPage[currentPage]=1&pageSize=12

hi! Please wait for the release of the next version
https://github.com/apache/incubator-shenyu/issues/3167


chore: bdd-gh-1586234488 to 0.0.1

chore: Promote bdd-gh-1586234488 to version 0.0.1

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approvers:
If they are not already assigned, you can assign the PR to them by writing /assign  in a comment when ready.
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


WandbLogger warning not logging logs.

🐛 Bug
WandbLogger giving warning: WARNING Adding to old History rows isn't currently supported. Step 25 < 38 and not logging when I try to use the WandbLogger with k-fold cross-validation because there I am using the same instance of wandb_logger but using trainer.fit multiple times for different train_dl and valid_dl. Since the step gets repeated in each case, it's not logging anything after the 1st fold is complete even though the log keys are completely different. It was working perfectly with pytorch-lightning v-0.7.4. For now, I have to create separate experiments for each fold which are hard to analyze on wandb.

To Reproduce
Code sample
Colab Notebook

Expected behavior
It should log even when the global_step is repeated in case if the logs keys are different.

Environment

PyTorch Version (e.g., 1.0): 1.5.0
OS (e.g., Linux): Linux
How you installed PyTorch (conda, pip, source): pip
Build command you used (if compiling from source):
Python version: 3.7
CUDA/cuDNN version: 10.1
GPU models and configuration: Tesla P100-PCIE-16GB
Any other relevant information:

Additional context

@borisdayma


fix hessian2 反序列化bug #3640

修复https://github.com/apache/incubator-dubbo/issues/3640 问题

Codecov Report

Merging #8 into master will increase coverage by 0.04%.
The diff coverage is 37.14%.

@@             Coverage Diff              @@
##             master       #8      +/-   ##
============================================
+ Coverage     24.24%   24.29%   +0.04%
  Complexity      455      455
============================================
  Files            89       89
  Lines          6289     6322      +33
  Branches       1048     1049       +1
============================================
+ Hits           1525     1536      +11
- Misses         4547     4568      +21
- Partials        217      218       +1

Impacted Files
Coverage Δ
Complexity Δ

...m/alibaba/com/caucho/hessian/io/Hessian2Input.java
19.43% <31.03%> (+0.39%)
62 <28> (ø)
:arrow_down:

.../com/caucho/hessian/io/CollectionDeserializer.java
66% <66.66%> (-1.4%)
12 <0> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5bcfcf4...4495e20. Read the comment docs.

Please see the java7 error https://travis-ci.org/dubbo/hessian-lite/jobs/506137514

Please see the java7 error https://travis-ci.org/dubbo/hessian-lite/jobs/506137514

I will look later, because I don't have a java7 environment yet, would you give me some advice?

I will look later, because I don't have a java7 environment yet, would you give me some advice?

I think you can setup a java7 in your pc
Additional,  please add some test case to verify your change for the issue: https://github.com/apache/incubator-dubbo/issues/3640

I will look later, because I don't have a java7 environment yet, would you give me some advice?

I think you can setup a java7 in your pc
Additional, please add some test case to verify your change for the issue: apache/incubator-dubbo#3640

Yes,I will add test case later.

I will look later, because I don't have a java7 environment yet, would you give me some advice?

I think you can setup a java7 in your pc
Additional, please add some test case to verify your change for the issue: apache/incubator-dubbo#3640

I tested it in java7 in my local environment and found that it was passed. I don’t have any ideas for now. Let’s take another look.

LGTM

这逻辑根本不对，多线程的时候，直接报错。


Changelog 0.5.1 / 0.6

https://github.com/dmacvicar/terraform-provider-libvirt/pull/444 ( add -version flag)
..  (compressed image support WIP)
..

Closing this in favour of a release draft (which already exists for 0.5.2)

@dmacvicar ok thx i didn't saw it.. :+1:


Refactor code to have Gradle source structure and write Gradle scripts

This is probably one of the most important tickets for making development easier and getting more people involved.

50053aed07b9f8daf7d4bfa3aaba6cdb390b6919 does this!


Create difference iterator

Creates an iterator to traverse the differences between two tree states, and corresponding tests.
Implements https://github.com/cerc-io/issue_tracking/issues/35
Pending:

[x] single-pass implementation
[ ] comparative benchmarks.

Single-pass implementation is now working. Benchmarks are not that meaningful without testing against a realistic consuming workload (i.e. DB updates), so I won't add them here.


getPath failed on empty paths

Changed head to headOption to prevent crash on empty paths

@mgttlinger thanks for the PR; one small comment :-)
/cc @kaiserpelagic

Sure thing, gonna change that later today.

What is your release schedule? Aka when can I expect a release with that PR?

@mgttlinger releases are automated on merge: https://repo1.maven.org/maven2/io/verizon/quiver/core_2.11/5.5.15-scalaz-7.1/
https://repo1.maven.org/maven2/io/verizon/quiver/core_2.11/5.5.15-scalaz-7.2/
etc

That's great!


When Follow task is given, actual task is not executed

Describe the bug
setup:
    image: golang
    commands:
     ....
build:
  - follow: setup
    image: golang
    commands:
    .....

Currently, it runs setup task first and not run build task at all after that.
Expected behavior
This is expected to run setup task first. If if fails, it should not continue. If it passes it should continue execution of build task.

@apoorvam I checked on my local machine using the following as the dunner file
build:
  - image: node:10.15.0
    commands:
      - ["printenv"]
    envs:
      - FOO_USER=foo
  - follow: update
    image: node:10.15.0
  - image: node:10.15.0
    command: ["npm", "--version"]
update:
  - image: ubuntu
    command: ['apt', 'update']

It works fine.
Note: While writing the feature, I assumed that no command will be passed in the follow step. Therefore, the following is not supposed to work.
  - follow: update
    image: node:10.15.0
    command: ["node", "--version"]

Without thinking on how current implementation is, please formulate on how this feature should look like. We can proceed with implementation post that.


Dev - create validators for all outgoing shippo objects

this isnt necessary. if missing or invalid data is sent to shippo endpoint it will respond with the error details...


Overview map keeps making requests

When I open the overview map control it keeps sending requests to the map server for some reason.
I've set it so it doesn't zoom in or out with the map, and it seems to stop the moment I've zoomed in a little bit. But when I zoom back out to my full view it just starts again.
Seems like a bit of overkill just for a little extra map in the bottom of my screen :)

Please show us in a jsFiddle, I cannot reproduce this in our examples.

I'm not able to reproduce it quickly outside our code either. I'll test some more and try to get back to you by monday!

see #8290


Bump Travis OTP version

Hi
A small update to bump the OTP version.
Thanks
Sam

Hello,
Another update with OTP 20

Sorry for the delay and thank you!


Update link to Nate's twitter account

@natebenes is my old, private twitter account.  Let's link to the public one instead.

👍


Is there a way to remove the TOC from an article doctype?

You just need to unset the toc attribute.
= Document Title
:!toc:

(Unlike other converters, in PDF, the toc is enabled by default).
For details, see https://asciidoctor.org/docs/user-manual/#user-toc

Doh!
Thanks

Except...  it doesn't work.  The pdf still has a table of contents.
:authors: author one, author two
:date: 2018-07-20
:doctype: article
:doctitle: The Title
:!toc:
:revnumber: 1
:linkattrs:
:imagesdir: images
:stylesdir: styles
:linkcss:
ifdef::backend-pdf[:notitle:]
ifdef::backend-pdf[]
[discrete]
= {doctitle}
endif::[]

Are you sure the command calling Asciidoctor is not passing the toc
attribute? Have you also verified these are the very first lines of the
file and they are followed by at least one blank line?
To check if the toc attribute is set, you can add this to the body.
{toc}
or
ifdef::toc[toc is set]

You nailed it... the gradle build had a toc attribute.

Yee-ha!


Improve speed port lookup in backendhttpsettings

Describe the bug
Currently we have 34 ingresses and services defined with each ingress having 2 hosts and 2 path routes.
When we change or add a route to the config it takes over 2 minutes to get the new route live.
We've noticed lots of the same info logs (I0911 14:12:10.048335       1 backendhttpsettings.go:159] resolving port name stats) giving us an idea there are way too many nested loops.
Is there a way to enhance this and improve the speed of such a tiny change?
The performance of the gateway is drasticly slow when more routes are added...
To Reproduce
Steps to reproduce the behavior:

Create a new path-based route
wait for more than 2 minutes
This timing will be increased the more routes and rules you add.

Ingress Controller details
Name:           ingress-azure-f66858b6b-zbm45
Namespace:      default
Priority:       0
Node:           aks-agentpool-75099453-0/<IP_ADDRESS>
Start Time:     Mon, 02 Sep 2019 16:00:21 +0200
Labels:         aadpodidbinding=ingress-azure
                app=ingress-azure
                pod-template-hash=f66858b6b
                release=ingress-azure
Annotations:    <none>
Status:         Running
IP:             <IP_ADDRESS>
Controlled By:  ReplicaSet/ingress-azure-f66858b6b
Containers:
  ingress-azure:
    Container ID:   docker://754e88caf6e7a6227b2a0c6a09b4c2be98cf13fdb21e2308454d172dea681dd3
    Image:          mcr.microsoft.com/azure-application-gateway/kubernetes-ingress:0.8.0
    Image ID:       docker-pullable://mcr.microsoft.com/azure-application-gateway/kubernetes-ingress@sha256:9d0e655368098dc64ef00cc93b920df8c959027a167ca82ef7b8408fca45bd3a
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Mon, 02 Sep 2019 16:00:24 +0200
    Ready:          True
    Restart Count:  0
    Environment Variables from:
      ingress-azure  ConfigMap  Optional: false
    Environment:     <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from ingress-azure-token-2fqnv (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  ingress-azure-token-2fqnv:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  ingress-azure-token-2fqnv
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

Output of kubectl logs <ingress controller>.
log.txt

@Baklap4 This is great feedback!
I'm going to take a look at that ASAP.
You pointed me to a function within our experimental Istio controller. Are you actually using this Ingress Controller within an Istio installation?
If not - the relevant code that emits this message is here: https://github.com/Azure/application-gateway-kubernetes-ingress/blob/0.8.0/pkg/appgw/backendhttpsettings.go#L159
Would you be so kind to provide a sanitized version of one of these 34 ingresses?  I'd love to simulate the environment in which you are running this.

Hi @Baklap4
I tested a scenario similar to what you are describing. As you pointed out with version 0.8.0 creating App Gwy config is indeed slower than it could be.
We just released version 0.9.0-rc2 - this utilizes memoization on functions that are often used and so I noticed significant improvement in time it takes to gen the config.
I do think we can do better in the area of the code you pointed out, and started work on this: https://github.com/Azure/application-gateway-kubernetes-ingress/compare/delqn/http_settings_perf?expand=1
I think 0.9.0-rc2 is already fast.
Would you be able to test 0.9.0-rc2 and report back?
Also regarding this, or any other areas of the codebase that you think could be improved - PRs are more than welcome!
I'll keep this issue open, but would love to close it when you confirm that the latest version is faster for your setup!

Heey @delqn
We've tried out v0.9.0-rc2 and it's indeed way faster and a lot less cpu usage. We were hovering around 40-50% cpu usage and are currently with everything deployed around 0-3%.
However due to upgrading to v0.9.0-rc2 we had to update our tls configuration. We have multiple namespaces and defined one tls config in namespaceA while ingress in namespaceB was also using this tls configuration but was not defined:
IngressA in namespace A
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: azure/application-gateway
  creationTimestamp: "2019-09-04T13:30:38Z"
  generation: 1
  labels:
    app: other-controller
    chart: app-0.2.7
    heritage: Tiller
    release: other-controller
  name: other-controller
  namespace: namespaceA
  resourceVersion: "2799276"
  selfLink: /apis/extensions/v1beta1/namespaces/namespaceA/ingresses/other-controller
  uid: 2ec69625-cf18-11e9-9b54-4e9e722f0d3c
spec:
  rules:
  - host: test.domain.net
    http:
      paths:
      - backend:
          serviceName: other-controller
          servicePort: 5000
        path: /other-path
  tls:
  - hosts:
    - test.domain.net
    secretName: domain-tls

status:
  loadBalancer: {}

Ingress B in Namespace B
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: azure/application-gateway
  creationTimestamp: "2019-09-04T13:30:38Z"
  generation: 1
  labels:
    app: differ-controller
    chart: app-0.2.7
    heritage: Tiller
    release: differ-controller
  name: differ-controller
  namespace: namespaceB
  resourceVersion: "2799276"
  selfLink: /apis/extensions/v1beta1/namespaces/namespaceB/ingresses/differ-controller
  uid: 2ec69625-cf18-11e9-9b54-4e9e722f0d3c
spec:
  rules:
  - host: test.domain.net
    http:
      paths:
      - backend:
          serviceName: differ-controller
          servicePort: 5000
        path: /differ-path
status:
  loadBalancer: {}

The domain-tls secret was defined in namespaceA.
This was a valid configuration for v.0.8.0 however when upgrading to v.0.9.0-rc2 this resulted in a bad gateway with the message:
RedirectWithNoTLS' ignoring Ingress namespaceB/differ-controller as it has an invalid spec. It is annotated with ssl-redirect: true but is missing a TLS secret. Please add a TLS secret or remove ssl-redirect annotation

Since secrets can't be shared between namespaces as per kubernetes ingress spec we had to copy our secret to namespaceB and configure a tls section.
Copying the secret was done by this command:
kubectl get secret domain-tls --namespace=namespaceA --export -o yaml | kubectl apply --namespace=join-data-namespaceB -f -

The ingress for Ingress2 looks now like this to make it work:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    appgw.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/ingress.class: azure/application-gateway
  creationTimestamp: "2019-09-04T13:30:38Z"
  generation: 1
  labels:
    app: differ-controller
    chart: app-0.2.7
    heritage: Tiller
    release: differ-controller
  name: differ-controller
  namespace: namespaceB
  resourceVersion: "2799276"
  selfLink: /apis/extensions/v1beta1/namespaces/namespaceB/ingresses/differ-controller
  uid: 2ec69625-cf18-11e9-9b54-4e9e722f0d3c
spec:
  rules:
  - host: test.domain.net
    http:
      paths:
      - backend:
          serviceName: differ-controller
          servicePort: 5000
        path: /differ-path
  tls:
  - hosts:
    - test.domain.net
    secretName: domain-tls
status:
  loadBalancer: {}

do you have any suggestions how to do this better or is this is an 'okay-ish' solution? I liked the idea to have one secret which could be used as configuration for other ingresses.. but somehow that seems impossible now..

@Baklap4 You can use the new annotation appgw-ssl-certificate to store the certificate on Application Gateway directly and referencing it in the ingress.


fixed to call janitor run method

https://github.com/Code-Hex/go-generics-cache/pull/26#issuecomment-1250216337

Previously, the janitor held in New was not stopped because SetFinalizer was not called. Therefore I have changed the specification in 9790bda.
I explain here why it is not stopped.

The janitor is stopped when the cache is not referenced anywhere.
The cache is referenced in the janitor run method.
The cache is always referenced. So It will be not called a finalizer.

If you want to stop the janitor gracefully, You must use the NewContext function.


[docs] Adopt new Pulsar Functions doc

Fixes #4554
Motivation
Adopt new structure for Pulsar Functions.
Modifications

Remove the old version docs;
Adjust the structure in sidebars.json file.
check and resolve link issues.

@sijie @wolfstudy PTAL, thanks.

run integration tests
run java8 tests

Got it, thank you for your reminder. I'll update those in 2.4.1 folders as well.


Support for LoongArch64

Please ping me when it's ready for review. :)

RFR, thanks @hyperxpro

You need to add a native module of LoongArch64 for the project to build. Also, update the CI workflow file for running tests.

You need to add a native module of LoongArch64 for the project to build. Also, update the CI workflow file for running tests.
I don't know if you have a loongarch machine, If there is no machine, does the CI workflow file not need to be modified?

Use QEMU in Docker. :)

Use QEMU in Docker. :)

I'm not quite sure how to modify CI workflow, is it possible to merge this patch into the repo? You then make changes to the CI workflow file？

Unfortunately, I can't merge until I have CI passing on this architecture.

I am not familiar with how to modify CI workflow, I may need your help. Thanks

Alright, I will try this.

Alright, I will try this.

Thanks


Link is broken when going back from AVS-Labs page to AVS main page

The link is broken going back from Table of Content to AVS main page, the reason is the extra forward slash appended to the URL.
This is the problem (notice the forward slash at the end)
https://microsoft.github.io/PartnerResources/azure/infrastructure/azure-vmware-solution/
This is supposed to be the solution:
https://microsoft.github.io/PartnerResources/azure/infrastructure/azure-vmware-solution

In addition, in the breadcrumb link, it should have vmware as: VMware

There are a couple of challenges here. Fundamentally, the breadcrumbing is designed to work within the defined toc.yml in the data folder. If there are additional folders it doesn't know about (as is the case here) rendering can be wrong -- it won't know proper case, etc. I moved this functionality to _includes/breadcrumb.html for future work to improve this. One possible solution is to add the folder structure to the lab in toc.yml, and modify breadcrumb.html to work recursively.
Another possible solution is to simply add "showbreadcrumb: false" in the front matter (top of page, like w/ permalink and tags). That removes the breadcrumb from the page, as it's not needed on some pages.
The best solution, however, I think is to move the lab to the new hands-on / skilling section, which is designed for workshop/lab content where there are subpages.
I'll discuss these w/ you. 👍


Added pprof profiling to monitor heap memory

Codecov Report

Merging #318 (890412a) into master (9f8d9b6) will decrease coverage by 1.84%.
The diff coverage is n/a.

:exclamation: Current head 890412a differs from pull request most recent head 3cdd82f. Consider uploading reports for the commit 3cdd82f to get more accurate results

@@            Coverage Diff             @@
##           master     #318      +/-   ##
==========================================
- Coverage   43.57%   41.73%   -1.85%
==========================================
  Files         121      121
  Lines        6240     6240
==========================================
- Hits         2719     2604     -115
- Misses       3319     3449     +130
+ Partials      202      187      -15

Flag
Coverage Δ

integration
?

unittests
41.73% <ø> (+0.22%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

pkg/messagebroker/factory.go
33.33% <0.00%> (-33.34%)
:arrow_down:

pkg/messagebroker/kafka.go
10.43% <0.00%> (-29.86%)
:arrow_down:

internal/subscriber/model.go
33.88% <0.00%> (+1.65%)
:arrow_up:

internal/stream/stream.go
33.01% <0.00%> (+5.66%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9f8d9b6...3cdd82f. Read the comment docs.