[WFLY-15461] Exclude jasypt from Galleon WildFly plugin transformatio…

…n during build/use of the wildfly-ee feature pack
https://issues.redhat.com/browse/WFLY-15461

@jimma Please review. I think the main review would be looking at my analysis in the WFLY-15461 description and see if you agree.

@bstansberry  This looks good to me . I saw there are only javax package name in these classes, but wss4j or cxf doesn't use them :
./src/main/java/org/jasypt/web/pbeconfig/WebPBEInitializationContextListener.java:import javax.servlet.ServletContextEvent;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEInitializationContextListener.java:import javax.servlet.ServletContextListener;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigFilter.java:import javax.servlet.Filter;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigFilter.java:import javax.servlet.FilterChain;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigFilter.java:import javax.servlet.FilterConfig;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigFilter.java:import javax.servlet.ServletException;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigFilter.java:import javax.servlet.ServletRequest;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigFilter.java:import javax.servlet.ServletResponse;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigServlet.java:import javax.servlet.ServletException;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigServlet.java:import javax.servlet.http.HttpServlet;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigServlet.java:import javax.servlet.http.HttpServletRequest;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigServlet.java:import javax.servlet.http.HttpServletResponse;
./src/main/java/org/jasypt/web/pbeconfig/WebPBEConfigHtmlUtils.java:import javax.servlet.http.HttpServletRequest;


Feature request: shared storage with OS X

In my team we are using mk8s for local development. It works great because we can use host storage to mount source code directly into containers and edit code with all the benefits of hot reloading.
Unfortunately, that only works for Linux users and we can't run same setup on OS X. It would be truly great if we could use microk8s + multipass to achieve same result. Perhaps it is possible to configure some sort of nfs or sync to share from host to VM and mount into k8s?
There have been other approaches to address similar issues:
http://docker-sync.io/
https://github.com/vapor-ware/ksync
https://github.com/GoogleContainerTools/skaffold
https://devspace.cloud/docs/cli/development/configuration/file-synchronization
https://github.com/okteto/okteto
https://stackoverflow.com/questions/47894378/is-it-possible-to-mount-a-local-computer-folder-to-kubernetes-for-development-l
I think this would be a great addition to microk8s and would help to adopt it outside core Linux users.

@tvansteenburgh @ammarn911 here are some interesting suggestions we could consider for the future.


Implement Various Utility Functions and Add Robust Input Handling

This PR introduces several utility functions and improvements, covering a variety of functionality. Key additions include:

sum_unique_elements: A function to compute the sum of unique numeric elements in a list while ignoring non-numeric values. Test cases handle edge cases such as empty lists, None values, and repeated numbers.

is_armstrong: A function to check if a number is an Armstrong number, with a main function to demonstrate user input validation and results.

fibonacci: Generates the first n Fibonacci numbers with input validation, handling edge cases for invalid or small inputs.

longest_substring_without_repeating_characters: Finds the length of the longest substring in a string without repeating characters. The implementation includes handling of edge cases such as empty strings or strings with repeated characters.

is_palindrome: Checks if a string is a palindrome, ignoring non-alphanumeric characters and case sensitivity. Added input handling for string normalization.

is_prime and sum_of_primes: Added prime-checking functionality and the ability to calculate the sum of all prime numbers up to a given limit, with robust input validation for numeric inputs.

reverse_words: Reverses the order of words in a sentence, with a main function to handle user input and demonstrate functionality.

These improvements ensure enhanced user input validation, better handling of edge cases, and test cases for each function, improving overall functionality and usability.

@LungsomLamnio  if you are interested in contributing this repo , feel free to do


Stockholm I/O: invitation for input on dealing with reference tags

Thanks @squirrelo for bringing this to my attention. In Stockholm, tree tags are handled differently than normal tags, as described here. This is the only tag type that has a different rule associated with it, or so I thought. There is another type of tag; reference, which appears to have its own rules as well. @jairideout and I have both combed the web as much as we can to try to find these rules, but as there isn't very much information on Stockholm already, we couldn't find any info on these tags, especially how this metadata should be handled. I am opening this for discussion for any input regarding how we should store this metadata.
This is the example file's GF lines:
#=GF RN   [1]
#=GF RM   10094622
#=GF RT   The S box regulon: a new global transcription termination control system
#=GF RT   for methionine and cysteine biosynthesis genes in gram-positive bacteria.
#=GF RA   Grundy FJ, Henkin TM
#=GF RL   Mol Microbiol 1998;30:737-749.
#=GF RN   [2]
#=GF RM   12910260
#=GF RT   An mRNA structure that controls gene expression by binding
#=GF RT   S-adenosylmethionine.
#=GF RA   Winkler WC, Nahvi A, Sudarsan N, Barrick JE, Breaker RR
#=GF RL   Nat Struct Biol 2003;10:701-707.
#=GF RN   [3]
#=GF RM   12702767
#=GF RT   The riboswitch-mediated control of sulfur metabolism in bacteria.
#=GF RA   Epshtein V, Mironov AS, Nudler E
#=GF RL   Proc Natl Acad Sci U S A 2003;100:5052-5056.
#=GF RN   [4]
#=GF RM   15215334
#=GF RT   Comparative genomics of the methionine metabolism in Gram-positive
#=GF RT   bacteria: a variety of regulatory systems.
#=GF RA   Rodionov DA, Vitreschak AG, Mironov AA, Gelfand MS
#=GF RL   Nucleic Acids Res 2004;32:3340-3353.
#=GF RN   [5]
#=GF RM   16810258
#=GF RT   Structure of the S-adenosylmethionine riboswitch regulatory mRNA element.
#=GF RA   Montange RK, Batey RT
#=GF RL   Nature. 2006;441:1172-1175.

According to the Wikipedia article, this is what all the reference tags mean:
RN -- Reference Number
RM -- Reference Medline
RT -- Reference Title
RA -- Reference Author
RL -- Reference Location
RC -- Reference Comment
So it appears every reference has to be preceded by a RN tag. Other than that, it seems the order after RN is RM, RT, RA, RL, but nothing is certain as there is no documentation on these tags, besides what they mean.
Here's what it looks like right now if a user tries to read this file into a TabularMSA:
>>> from skbio import TabularMSA, RNA
>>> with open('Stockholm.txt', 'r') as fh:
...     msa = TabularMSA.read(fh, constructor=RNA)
>>> msa

TabularMSA[RNA]
-----------------------------------------------------------------------
Metadata:
    'AC': 'RF00162'
    'AU': 'Griffiths-Jones SR, Gardner PP'
    'BM': 'cmbuild -F CM SEED'
    'CB': 'cmcalibrate --mpi CM'
    'CC': <class 'str'>
    'DE': 'SAM riboswitch (S box leader)'
    'DR': 'SO; 0000035; riboswitch;'
    'GA': '44.00'
    'ID': 'SAM'
    'NC': '43.90'
    'PI': 'S_box'
    'RA': <class 'str'>
    'RL': <class 'str'>
    'RM': '10094622 12910260 12702767 15215334 16810258'
    'RN': '[1] [2] [3] [4] [5]'
    'RT': <class 'str'>
    'SE': 'Grundy F, Henkin T, PMID:10094622'
    'SM': 'cmsearch --cpu 4 --verbose --nohmmonly -E 1000 -Z
<PHONE_NUMBER>50 CM SEQDB'
    'SQ': '433'
    'SS': 'Published; PMID:10094622'
    'TC': '44.00'
    'TP': 'Cis-reg; riboswitch;'
    'WK': 'SAM_riboswitch_(S_box_leader)'
Positional metadata:
    'SS_cons': <dtype: object>
    'RF': <dtype: object>
Stats:
    sequence count: 433
    position count: 231
-----------------------------------------------------------------------
UUUCUAUCCAGAG-AGG-U-GG----------- ... -------------GCU-UG-A-AA-GAUAGGAA
ACCUUAUUUUGAG-AAG-C-UG----------- ... --------------CU-CG-A-AU-GAUAAGUA
...
AACUUAUCACGAG-CGA-C-CG----------- ... -----------CGCUU-CG-G-AA-GAUAAGGA
GUCUUAUCCAGAG-CAG-C-GG----------- ... ------------UCCG-GG-A-AA-GAUAAGGA

Looks fine, but if we look at the RA metadata tag for example, we will see the data is being stored incorrectly:
>>> msa.metadata['RA']

Grundy FJ, Henkin TM Winkler WC, Nahvi A, Sudarsan N, Barrick JE, Breaker RR Epshtein V, Mironov AS, Nudler E Rodionov DA, Vitreschak AG, Mironov AA, Gelfand MS Montange RK, Batey RT

Instead of seeing separate authors, RA holds all the authors as one long, ugly string. This is because reference tags are being treated as default GF tags, and every time the reader sees another RA tag, it just appends its metadata to the string, as seen above. Additionally as an example, the RN tag, which should store the Reference Number(s), is simply a string:
'[1] [2] [3] [4] [5]'

I know this issue is a bit of a read, but this is all the info on the subject.
Any input is greatly appreciated!

It looks like skbio.io.format.stockholm.add_gf_metadata is handling a few special cases and then everything outside of those cases is either being created as a string, or if the feature already exists it is concatenating the new data onto the existing data, resulting in a single string of all of the data. It looks like all of the R_ data, or really all of the GF data that isn't handled in the special cases, is currently being concatenated into a single string.
To resolve this you could switch them to be stored as lists and append every new chunk of feature_data, but as seen in this example from your post:
#=GF RN   [4]
#=GF RM   15215334
#=GF RT   Comparative genomics of the methionine metabolism in Gram-positive
#=GF RT   bacteria: a variety of regulatory systems.
#=GF RA   Rodionov DA, Vitreschak AG, Mironov AA, Gelfand MS
#=GF RL   Nucleic Acids Res 2004;32:3340-3353.

Some single blocks of data have multiple lines of the same feature (specifically RT in this example), so appending onto a list would cause what was meant to be a single element of feature_data to be broken into multiple elements, so in order for lists to work you would also have to implement some sort of marker in the metadata addition to know which element of that feature you are working with. This is all assuming they are wanting to be accessed as separate items in a list though.

Thanks for commenting @jakereps, yes that is exactly what is happening.
That's a good idea for storing this data, I'm thinking it's going to have to be something similar to the way we store trees, documented here.
Maybe some sort of nested OrderedDict?

I'm not familiar enough with how these are used to speculate on which way is the best way, but they could either be stored as each Reference in a single object with each RN as the key, or keep them all as their separate RN, RM, etc... keys with the RN being the key to the data for each, so something along the lines of:
msa.metadata['REFS']['[1]']['RA'] // this seems pretty heavy
metadata = {
    'REFS': {
        '[1]': {
            'RM': '10094622',
            'RT': 'The S box regulon: a new global transcription termination control system for methionine and cysteine biosynthesis genes in gram-positive bacteria.',
            'RA': 'Grundy FJ, Henkin TM',
            'RL': 'Mol Microbiol 1998;30:737-749.'
        }
    }
}

msa.metadata['RA']['[1]']
metadata = {
    'RN': ['[1]', '[2]'],
    'RM': {
        '[1]': '15215334',
        '[2]': ...
    },
    'RT': {
        '[1]': 'Comparative genomics of the methionine metabolism in Gram-positive bacteria: a variety of regulatory systems.',
        '[2]': ...
    },
    'RA': {
        '[1]': 'Rodionov DA, Vitreschak AG, Mironov AA, Gelfand MS',
        '[2]': ...
    },
    'RL': {
        '[1]': 'Nucleic Acids Res 2004;32:3340-3353.',
        '[2]': ...
    },
}

Thanks for the discussion on this @Kleptobismol and @jakereps!
Since the reference number isn't all that useful (it just assigns an increasing number to each reference), we could store the references as a list of dicts (OrderedDict shouldn't be necessary here since the list maintains order for us). By storing the references in a list, users can access each reference by index position (basically the reference number, just 0-based indexing instead of 1-based). I think this will be more natural/Pythonic than accessing data by RN key (e.g., [1], [2]). Something like:
msa.metadata = {
    'RN': [
        {'RM': '10094622', 'RA': 'Grundy FJ, Henkin TM', ...},
        {'RM': '12910260', 'RL': 'Nat Struct Biol 2003;10:701-707.', ...},
        ...
    ]
}

msa.metadata['RN'][0] # access the first reference
msa.metadata['RN'][1] # access the second reference

This structure is basically an inversion of @jakereps structure above. I like this a little better because it keeps all the data about a reference in a single dict, making it easier to retrieve that info in a single command.
We could use a different tag than 'RN' to store the list of references, but it seems safer to use 'RN' since we know that it is a reserved tag with special meaning. If we use a different tag, e.g. 'REFS', that could overwrite data if the Stockholm file happens to have the tag 'REFS' pointing to something else (remember that these tags are arbitrary).
What do you guys think about this structure?

This sounds great, @jairideout! I think it's a good idea to store the data under the RN tag, because that way when they index in like this:
msa.metadata['RN'][0]

it will be like you are pointing from metadata --> Reference Numbers --> reference number, which seems intuitive for the user.

Oh, nice. Yeah that's definitely cleaner, @jairideout. I wasn't sure if the RN was being used somewhere else in the data, so was trying to hold onto it, but a pure list instead of ...['[1]'] is much easier to use. So for the single reference, but multiline feature data concatenation, you could just hold onto the RN you're currently at until either hitting a new reference number, or hitting something that doesn't start with R, or would it have to be more universal than that?

Well, the reference number is always stored under the RN tag, so any time the parser sees that, it will append a new dictionary to the list. Additionally, the dictionary will initially populate with empty strings, so anytime it sees a tag, for example RM, it will simply add it like this:
If the RM tag was seen and stored in feature_tag, and its data in feature_data:
self._metadata['RN'][-1][feature_tag] += feature_data

I really like the ['RN'][0] == {'RM': '0000000000', ...} idea, as it seems the cleanest of all proposed, and easiest to program.  From wikipedia it looks like only reference tags start with R, but that might not be a safe assumption in the future.

Regarding the top-level key, are there any symbols excluded by Stockholm we could use? I like the concept of refs or something like that a bit better than RN, but RN is a safe fallback I suppose.

I think using RN is a safe bet, especially since the index of the list sort of becomes the Reference Number.

@Kleptobismol if you padded the 0 index of the list upon creation, with either an empty string or a counter of total ref fences, then they could all match exactly as the index from the input, and could avoid any translation confusion between the 1 based RN and a 0 based Python list. Or is this mostly used by programmers where using 0 to get the [1] would make more sense?

@ebolyen I can definitely see where a refactor like this could make it strange to lump everything under the RN tag, where typing metadata['RN'] would dump all of the references and their data instead of just the reference numbers, and using refs or something similar as the top level could help with the readability.

@jakereps the reference number appears to only be a numbering on a list of references, not a unique id. I think the only reason it exists is to clearly separate the references in the Stockholm file. So, I don't think it matters to hold on to that data, it appears the only thing that needs to be maintained is the order.

Agree with @Kleptobismol, padding the list would also make the list far harder to deal with (polymorphic vs monomorphic). It is also very likely the the user doesn't have a particular reference they are interested in, if they knew, they would already have the data... so just collecting them together in an iterable should be more than sufficient.
Still feels strange to call it RN though:
reference_number = 0
msa.metadata['RN'][reference_number]

vs
ms.metadata['references'][reference_number]

In order to be sure the tag doesn't interfere with other tags, we could name it something more unique, like collected_references or something. Thoughts?

I agree it'd be ideal to store the references under a key with a more descriptive name. Stockholm doesn't have any exclusions to tag names except that they can't contain whitespace, so we could safely use a key that has whitespace (e.g., msa.metadata['reference list']). Having a key name with spaces in it seems likely to frustrate users, and it would be difficult to come up with an intuitive multi-word name for "references", so I'm not too keen on this solution.
I agree it's a little odd to store the references under the 'RN' tag, but it's probably the most likely place a user would look for references since we can't use a reasonable name (e.g., 'references'). Storing references under the "magic" tag 'RN' is also consistent with how we are storing trees under the "magic" tag 'NH'. A 'trees' tag might be more descriptive/intuitive but we are again limited by how arbitrary Stockholm files can be.

instead of creating arbitrary keywords, what about just store all those text as a string and let users process it whatever they like?

Can you post an example of how the data would be stored?

like the following, since in most cases, users don't need to do any compute on references, so it probably fine to just store them unparsed.
seq.metadata['reference'] = '''#=GF RN   [1]
#=GF RM   10094622
#=GF RT   The S box regulon: a new global transcription termination control system
#=GF RT   for methionine and cysteine biosynthesis genes in gram-positive bacteria.
#=GF RA   Grundy FJ, Henkin TM
#=GF RL   Mol Microbiol 1998;30:737-749.
#=GF RN   [2]
#=GF RM   12910260
#=GF RT   An mRNA structure that controls gene expression by binding
#=GF RT   S-adenosylmethionine.
#=GF RA   Winkler WC, Nahvi A, Sudarsan N, Barrick JE, Breaker RR
#=GF RL   Nat Struct Biol 2003;10:701-707.'''

As discussed above, we can't define a 'references' tag because Stockholm files are composed of arbitrary tags that could include 'references' (namespace conflict). Also, storing the references unparsed differs from how other Stockholm tags are stored in .metadata/.positional_metadata (see scikit-bio Stockholm docs for examples).

right, we can change reference for other keyword. My point is not to parse it. Just MHO.

Regardless of what name we pick for the key, it could have a namespace conflict because of how general the Stockholm format is. The issue isn't specifically with the name 'references'. I don't think it's a good idea to leave the references unparsed because it is inconsistent with how other tags are handled and will likely be unexpected to users.
Are there any objections to storing a list of parsed references under the 'RN' key? Based on the discussion in this thread, it seems to be a good compromise because the references can be parsed, and there is no tag namespace conflict because 'RN' has special meaning in Stockholm.

:+1:
On Wed, May 18, 2016 at 10:46 AM, Jai Ram Rideout<EMAIL_ADDRESS>wrote:

Regardless of what name we pick for the key, it could have a namespace
conflict because of how general the Stockholm format is. The issue isn't
specifically with the name 'references'. I don't think it's a good idea
to leave the references unparsed because it is inconsistent with how other
tags are handled and will likely be unexpected to users.
Are there any objections to storing a list of parsed references under the
'RN' key? Based on the discussion in this thread, it seems to be a good
compromise because the references can be parsed, and there is no tag
namespace conflict because 'RN' has special meaning in Stockholm.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/biocore/scikit-bio/issues/1347#issuecomment-220104505

👍

Marking this API proposal as accepted, supporting issue is #1348.


Add 'include_koji_repo' user parameter

Add a user parameter 'force_tag_repo' with the meaning "include the
build tag repository from koji even yum_repourls" is set. This will
be used by the atomic reactor resolve_composes plugin to avoid
suppressing the build tag repository in circumstances where that
isn't desired.
Signed-off-by: Owen W. Taylor<EMAIL_ADDRESS>Maintainers will complete the following section:

[x] Commit messages are descriptive enough
[x] "Signed-off-by:" line is present in each commit
[x] Code coverage from testing does not decrease and new code is covered
[x] JSON/YAML configuration changes are updated in the relevant schema
[x] Changes to metadata also update the documentation for the metadata
[ ] Pull request includes link to an osbs-docs PR for user documentation updates

The name seems a bit confusing to me: 'force tag' is a commonly used term. Maybe 'include_koji_tag'?

The name seems a bit confusing to me: 'force tag' is a commonly used term. Maybe 'include_koji_tag'?

I have no strong attachment to the name - just need to shorten "force_include_koji_target_build_tag_repository" to some manageable length. :-) include_koji_tag is sounds like it has something to do with tagging the resulting builds. Maybe include_koji_repo?

include_koji_repo sounds fine to me.

New version updates parameter name.

/packit copr-build

Re-alphabetized as necessary (got a little hasty with the search-and-replace when renaming :-)

/packit copr-build

no release notes as this is only support for atomic-reactor patch (atomic reactor includes release notes)


Version channel
Stable (Default)
What part of Adonis is this related to?
Command
What happened?
out of thin air all the people on my game are unbanned, why is that? literally years of banned people and now they are all poof now all unbanned, please fix this.
Relevant log output
out of thin air all the people on my game are unbanned, why is that? literally years of banned people and now they are all poof now all unbanned, please fix this.

that's a nice relevant log output

@perrapig By chance is your DataStoreName and Key the same as before?

And I'd also love to see version of main module run :changelog and version of loader

im sorry the mayus were enabled, let me try get the things you asking for sorry

is this what you asking for? and also i've never checked the data stores names or anything like that, i just remember banning and thats it, but im not a programmer so i don't really know what happened, i haven't touched any setting myself.

No clue why this would have happened

what your describing is impossible without someone touching the settings file
you are the first person ever to experience this problem

L


Please add support for Resharper 2016.1

Please add support for latest resharper.

+1, just upgraded and was sad to discover I've lost Stylecop

+1

+1

+1

do you have any upgrade plans?

Yes, it'll be out this week.

Thanks @citizenmatt!

+1

+1

+1

+1

+1

+1

+1

Sorry for the delay, everyone. I'm working on it, fixing a couple of bugs, it'll be out very soon.

Thanks, I am going to test it :)

Works like a charm - Super... Thx...

Yeah installed yesterday and working perfectly, thanks @citizenmatt!


Chisels and Bits incompatibility

As far as I know, these two mods are incompatible together, it's nothing as serious as a crash but any blocks 'chiseled' with the mod turns the block invisible immediately. After asking around in their discord server it's likely due to the way ImmPtl changes rendering. It could be a total game changer if compatibility was integrated and I'm sure the dev would be willing to work with you for it! Hell, I'd pay for compatibility purely so I can live in a tiny home. Please delete this if it's already been discussed elsewhere, I checked all active issues.

compatability would be nice, but you can also use the pehkui mod on its own, or another mod that uses pehkui

Have you tried bits and chisels?

Have you tried bits and chisels?

No, but I will now. I'm pretty sure it'll be a similar result though.

compatability would be nice, but you can also use the pehkui mod on its own, or another mod that uses pehkui

It has nothing to do with pehkui but the portals rendering changes themselves

Someone said that bits and chisels is compatible but chisels and bits is incompatible so it may not be similar

Someone said that bits and chisels is compatible but chisels and bits is incompatible so it may not be similar

The fabric version is compatible but the forge versions are not, which is what I'm trying to use.

Bits and Chisels (Fabric only)
Chisels and Bits (Forge and Fabric)


Public API to preempt tf.train.Server

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): N/A
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): N/A
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: N/A
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): v1.9.0-0-g25c197e023 1.9.0
Python version: N/A
Bazel version (if compiling from source): N/A
GCC/Compiler version (if compiling from source):
CUDA/cuDNN version: N/A
GPU model and memory: N/A
Exact command to reproduce: N/A

Describe the problem
tf.train_and_evaluate uses time.sleep to (optimistically) synchronize the startup of chief/worker nodes in the distributed mode (see estimator/training.py). The implicit assumption in this logic is that by the time the worker is spawned, the chief has already started its tf.train.Server, i.e. the scheduler should be aware of the assumption and should schedule and initialize the chief first. This might not be easily achievable on general purpose systems like YARN.
One possible solution to this is to synchronize the chief/worker tasks on a barrier, and then preemptively start the server right after the barrier, but prior to calling train_and_evaluate. This does not eliminate the race condition entirely but makes it much less likely in practice. The only problem here is that train_and_evaluate does not provide a documented way to account for preempted servers. The undocumented way is:
https://github.com/tensorflow/tensorflow/blob/2b4fd1c2b7a37367e61bbae3d27d194a894cb7bb/tensorflow/python/estimator/training.py#L747-L748
I was wondering if it would be possible to make this part of the train_and_evaluate contract public or, alternatively, address the issue in another way?

@xiejw @ispirmustafa -- what is the recommended way to handle pre-empted Servers for systems like YARN?

@xiejw thank you for the clarification, and apologies for poor wording on my side. The code fragment I've linked to gives the "delay starting" rationale in the comment.

All workers (including chief) will try to find each other first
Would it be right to say that in order for a task to start, it should successfully connect to all other tasks; and that if the connection is unsuccessful, the task will retry indefinitely/"long enough"? I would very much appreciate it if you could reference relevant code in the reply.

Assume device_filter is not set. The precise sequence can be described as follows:

start_std_server. This opens the port and listens the grpc in background thread. All workers must do this first. Otherwise it might delay other workers to train the model.

(optional) time.sleep for delay starting. Note that at this time the grpc port is open already.

Estimator.train. This creates the graph and launches a new tf.Session to train the model. The session will try to talk to each other device (in this case other workers) and then run the TF graph. This is the part it will try to block until all devices are found. The code is here.

Say there is a worker in the time.sleep for 100000 secs (basically forever). As the port has been opened in step 1, so other workers can start train (step 3) without waiting.
With device filter, only step 3 is slightly different where each worker only tries to find the devices in the list rather than all devices.

This is indeed a non-issue, thanks again @xiejw!

@xiejw I thought about this more over a weekend and came up with another use-case for preemption: port reservation.
YARN does not manage the network on the containers, so in order to run TF on YARN one has to manually reserve a port on each of the allocated containers, communicate it to all other tasks to assemble a ClusterSpec and only then start the server. IIUC with the current API there is no race-free way of doing this is in TF. Could you advise?

@karmel do we have any expert for YARN? I do not understand that environment, so it is very hard for me to give best suggestions here.
@superbobry As I mentioned above, I do not understand YARN. So, my suggestions could be wrong or sub-optimal. The minimal knowledge each training job should know is

The port itself uses
The ports used by PS

The ports of other workers are optional. You still need to fill the TF_CONFIG, but the values of other workers do not matter.
If the conditions above are met in YARN, then you can construct a device_filter (see example code). The session will try to find the device in that list only even other workers are online (or even the other worker's addresses are fake).

@xiejw the problem with YARN is that it does not manage the network, i.e. there is no way to reserve a port prior to submitting an application. One possible solution to this is to

Spawn a Python process in each YARN container.
Bind to port 0.
Communicate the port assigned by the OS to all other containers and aggregate a cluster spec
Export the spec in TF_CONFIG and start training.

which roughly translates to
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
    sock.bind(("", 0))
    ipaddr, port = sock.getsockname()
    cluster_spec = broadcast_addr_and_aggregate_spec(f"{ipaddr}:{port}")

# export TF_CONFIG
tf.estimator.train_and_evaluate(...)

Note that this implementation has a race condition between closing sock and binding a tf.train.Server to the reserved port. This means that a task from another distributed TF training running on the same YARN node could hijack the port, and effectively talk to a completely unrelated task. The window of opportunity for the race condition to occur is undefined and depends on the implementation of train_and_evaluate.
I see multiple potential ways of getting rid of the race condition by slightly modifying the existing Python/C++ API of tf.train.Server:

Add an API to spawn a server using an existing FD. The gRPC backend seems to already support this (see grpc/grpc#6610).
Allow SO_REUSEPORT when creating a server. Currently the option is explicitly disabled.

@mrry could you comment on the flakiness caused by SO_REUSEPORT?

Nagging Assignee @karmel: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

@jhseu -- I hear you might be able to advise on YARN?

Nagging Assignee @karmel: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

I don't have much personal experience with running on YARN, but the LinkedIn folks do:
https://github.com/linkedin/TonY
https://engineering.linkedin.com/blog/2018/09/open-sourcing-tony--native-support-of-tensorflow-on-hadoop

Nagging Assignee @karmel: It has been 15 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @karmel: It has been 29 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

Nagging Assignee @karmel: It has been 47 days with no activity and this issue has an assignee. Please update the label and/or status accordingly.

DistributionStrategies is the new API for handling distribution and synchronization, and that API will be preferred in the coming months to the tf.train.Server API. That said, we have now started a Special Interest Group (SIG) for networking issues like these, so while I am closing this issue as it pertains to tf.train.Server, I would encourage you to join and participate in the Networking SIG to ensure these questions are addressed for YARN and other systems.


Symbol support with symbol overrides

Currently there appears to be no way to deal with pulling content from the spreadsheet using the symbol overrides.

Hi @custa1200 , Did you see the version 0.3.1? That has new symbol support. Is that what you are looking for?

@custa1200, were you able to look at the new symbol support? Would love your opinion on it.
This issue will be closed at the end of the week unless there are additional comments here.

Hey @dbrody I'll check it out agin soon but I am pretty sure that I was unable to get it to work with nested symbols last time I tried. I'll put together a small file later today to test it out how I would be using it.

That would be great. I don't think it currently supports nested symbols, so seeing an example of how you may use it would be helpful.

On Nov 28, 2016, at 5:07 PM, Mark<EMAIL_ADDRESS>wrote:
Hey @dbrody I'll check it out agin soon but I am pretty sure that I was unable to get it to work with nested symbols last time I tried. I'll put together a small file later today to test it out how I would be using it.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

Since this issue is on symbol overwrites which are supported now, we are closing this issue.
This topic can be continued for Supported Nested Symbols in issue #12


Redirect to Form landing page

Per the comment in source, it would be better to redirect to the form success page instead of rendering it on post.
A simple way to achieve that without needing to worry about adjusting url routes and such would be to use a query param... "/foo-form/?success".
If https://github.com/torchbox/wagtail/issues/2833 is accepted, the FormSubmission instance id or randomized token could be passed as well so it can be fetched and passed to the template.
If security is a concern around putting the id/token in url and leaking data to malicious users, it shouldn't be very difficult to use session data for verifying the user created the FormSubmission data or raising http404.
-- edit --
Forgot to mention another practical use for the query param (or unique url but more complicated with wagtail page routing) is it makes conversion tracking simple because you can simply check destination instead of needed developer to trigger an analytics event specific to the submission.

Hi I am new to contibuting and programming. I've read that the best way to learn is to contribute so I did redirection as suggested above using query param but a few tests started to fail. Most of them fail because of AssertionError: Couldn't retrieve content: Response code was 302 (expected 200). I see that it is due to the redirection but how to get rid of it. Should I adjust tests or my new code?

Hi @karabellyj - thanks for taking this on!
Adjusting the tests to reflect the new behaviour is the right thing to do here.

Solved that error by adding follow=True to the tests but now CustomSubmission tests are raising AssertionError: False is not true : Couldn't find 'Thank you for your patience!' in response and I don't see why.

I've created two pull requests to address this issue:
https://github.com/wagtail/wagtail/pull/4024  is a m one minor change that I think should be merged anyway. It simply moves the render-part of the serve method for the landing page to a separate method so that you can override the rendering of the landing page without having to copy paste all the other stuff from the serve method.
https://github.com/wagtail/wagtail/pull/4025 is up for discussion as this adds a new optional field to the model. (And I now realize I didn't make any migrations for this, yet). This new field is a foreign key to a page; that if set, is is used to redirect to instead of rendering the landing page.

I also made another PR that implements the success parameter idea: https://github.com/wagtail/wagtail/pull/4026
To fix the CustomSubmission test errors, I  had to modify the model FormPageWithCustomSubmission, slightly so that it doesn't apply it's overrides in the render method if the success param is set.

@strindhaug & @gasman
We need a way to ensure that the form_submission instance can be passed to the context of the landing page.
As per this outstanding item in #2833

The FormSubmission instance should then be passed to the render context for the "landing page".

Suggested Approach (without redirect)
Assuming we go with PR #4024 (which is not the redirect approach), the render_landing_page method should also accept a parameter form_submission.
Which should be generated by updating the line:
self.process_form_submission(form)
to
form_submission = self.process_form_submission(form)
Then
return self.render_landing_page(request, *args, **kwargs)
to
return self.render_landing_page(request, form_submission=form_submission, *args, **kwargs)
Finally, the context should be updated in the render_landing_page return render eg.
 # TODO: It is much better to redirect to it
context = self.get_context(request)
context['form_submission'] = form_submission
    return render(
        request,
        self.get_landing_page_template(request),
         context
        )

Closing as redirecting does not provide a reliable way to send context and redirecting with url params will be different for every use case.
Instead #4024 provides a way for developers to override the render_landing_page.

I came across this issue trying to accomplish something similar. In my situation, I'm wanting to change the URL for tracking purposes through Google Tag Manager rather than redirecting for some other purpose. My solution to this is actually using the RoutablePageMixin with the form page and creating a new routeable method for the submission. Inside the form template, I then set the action of the form to the submit route so the URL changes when the form is submitted. My issue now, is that the URL changes even if the form isn't valid... which is not quite what I want, but its a step in the right direction! I'm just hoping this will help someone else looking for something similar.
`class MyFormPage(AbstractEmailForm, RoutablePageMixin):
@route(r'^submit/')
def submit(self, request, *args, **kwargs):
    return self.serve(request, *args, **kwargs)`

`
`


Use snake case format for the log properties

In correlation with the goa generated logs, we want to use snake case properties for our logs.

@alexeykazakov we need to follow the same pattern to be able to consume and query the properties when analyzing the logs, or when generating dashboards.


fix: data collection on SHC

This fix was tested to work on Noah and Classic architectures.
Similar situation (in 2016...): https://community.splunk.com/t5/All-Apps-and-Add-ons/Splunk-Add-on-for-Tenable-Why-is-the-add-on-not-able-to/m-p/230462

:tada: This PR is included in version 6.0.5 :tada:
The release is available on:

v6.0.5
GitHub release

Your semantic-release bot :package::rocket:


Write documentation

I would make this issue into a milestone, so we can start refining the tasks and distribute the load.

This issue didn't help. ;)


Cannot read property 'offsetHeight' of undefined

Hi There!
I was recently given a site that contains this script, https://www.urac.org/directory-accredited-organizations .  However, it doesn't appear to be working at in the console log i see:
Uncaught TypeError: Cannot read property 'offsetHeight' of undefined
at lib.Child. (pym.v1.js:942)
at new lib.Child (pym.v1.js:1098)
at newdirectory?initialWidth=2560&childId=directory-iframe&parentTitle=Directory+of+Accredited+Organizations&parentUrl=https%3a%2f%2fwww.urac.org%2fdirectory-accredited-organizations:46
The code as i see it implemented:
Parent Page:
<script src="https://pym.nprapps.org/pym.v1.min.js"></script>
<script>
  var pymParent = new pym.Parent('directory-iframe', 'https://accreditnetadmin.urac.org/UracPortal/directory/newdirectory', {});
</script>```

Child Page:

```<script src="https://pym.nprapps.org/pym.v1.js" type="text/javascript"></script>
<script>var pymChild = new pym.Child();</script>```

In reading over the docs, it appears that everything was setup correctly, but I'm unsure since I've just been given this assignment.

Any insight would be great.

Thanks

Your page loads and initializes Pym inside the head tag, which means that when it tries to find and send the height of the body tag, the browser's HTML parser hasn't created the body yet, and the script fails. Move the Pym-related code in the child page to the bottom of the body tag to give the parser time to work.

Thanks for this.  It did fix my issue, however my child iframe is still not resizing.  I put the following code on the child to manually send the height back to the parent but it doesn't appear to be doing anything. Any thoughts?
var pymChild = null;

            document.addEventListener("DOMContentLoaded", function () {
                pymChild.sendHeight();
            });

            pymChild = new pym.Child();

Thanks!

I usually turn on the { polling: 200 } option for Pym children instead of manually sending the height on each DOM change. If that doesn't solve your problem, it may be worth making sure that the body of the child page has a measurable offsetHeight--in some cases, CSS changes that set a percentage height will cause the element to size itself to the window, and not vice-versa.


Implement ES2018 Async Iteration (Experimental)

This PR implements ES2018 Async Iteration (Async Generator Functions and for await):
UPDATE: This is now a complete implementation except:

It needs more tests (help appreciated here)
I haven't looked at profiler support
I haven't looked at TTD support
This is placed behind a flag, ES2018AsyncIteration set to default to disabled  (hoping it can be enabled soon but feels too big to enable straight away)

Implementation notes:

AsyncGenerators are implemented by extending the existing Generator type
Handling await, yield and yield* in async generators has involved adding 4 new op codes - wanted it only to be 3 but couldn't find a way to handle the interaction between .return() and yield* without an extra op
AsyncFromSyncIterator (required per spec for async iteration of a synchronous object) has been added as a whole new type

Test262 results

I've run this against the test 262 tests in the following folders, I used  the ES6FunctionNameFull and ES2018ObjectRestSpread flags when running the tests as some of these tests require those features:
https://github.com/tc39/test262/tree/master/test/built-ins/AsyncFromSyncIteratorPrototype
https://github.com/tc39/test262/tree/master/test/built-ins/AsyncGeneratorFunction
https://github.com/tc39/test262/tree/master/test/built-ins/AsyncIteratorPrototype/Symbol.asyncIterator
https://github.com/tc39/test262/tree/master/test/language/statements/async-generator
https://github.com/tc39/test262/tree/master/test/language/statements/for-await-of

Most tests pass, fails relate to existing issues:
a. Next isn't cached - issue exists with synch iterators https://github.com/Microsoft/ChakraCore/issues/5860
b. the async keyword is not allowed to contain unicode escapes (affects normal async functions as well as async generators)
c. of in a loop header is not allowed to contain a unicode escape
d. 12 other issues to do with destructuring syntax and loop headers

ref: #2720

Thanks, @rhuanjl . You mention that you're adding runtime checks. Do those checks determine which context await/yield are being employed in, and/or to distinguish between await and yield?

The runtime checks this contains which could clearly be removed by refactoring (likely with more code in total) are:

await vs yield vs yield*
AsyncGenerator vs Generator (when calling the next/throw/return methods have to check type twice as first check gives same answer for both as it's the same class)

I see. Yes, you're right, we want to be able to resolve static properties of the code without using runtime checks. A separate opcode for yield in different contexts is a good place to start and something we anticipated having to do to implement the feature. I'll be glad to help you do that. We can handle the JIT after getting things working in the interpreter. You'll need to:

add your new opcode to runtime/bytecode/opcodes.h
make the byte code generator emit your opcode in the proper context (runtime/bytecode/bytecodeemitter.cpp)
add a handler for your opcode to the interpreter, which will likely just route to a runtime helper.

Whilst this version should work as is certainly agree that it could do with improvements, I'll aim to submit an update with additional opcodes by the end of this week or start of next.

I don't understand the assertion this hits - my guess is it is to do with my new OpCodes either:
a) Using the ResumeYieldData from the stack OR
b) Using the RHS value without copying it to a temporary register
It runs fine in a test build so is it just an attribute I need to set somewhere to avoid the assertion?
Changes made:
I've made the following changes from the original version:

fixed some errors I'd made around incorrect handling of returns
Introduced OpCodes for Await, AsyncYield and AsyncYield*
Implemented the OpCodes as runtime function calls
added the OpCodes as helper calls in the Jit (I think I have this right...)
Removed most of the temporary objects my initial version needed (including pre-creating and holding onto continuation functions for await and yield as with most async generators these will likely be needed multiple times)
Significantly reduced the number of runtime checks/branches required
Updated my handling of await for a normative PR that's currently open to the spec (it already has consensus so should in theory be being merged soon)

OtherNotes

Hopefully the assertion I'm hitting is an easy fix though I'd appreciate help with that - I've had a look but can't figure it out.
If/when this is merged I'd like to follow up with an edit to the Async function implementation to use much of this code - will be able to reduce a lot of duplication as well as streamlining async functions
Pending points from my first comment remain the same

Small update - I'm about half way there offline with the remaining pieces needed to meet the spec i.e. for-await-of and async from sync iterator. It's going to be 500+ more lines of code when done so may be easier if this part is reviewed first before I add the rest?

Thanks, @rhuanjl . Are you still hitting the assert? If so, can you give me some detail about it?

@rhuanjl I mentioned this in private already but it seems that this feature currently can be enabled only with a flags-enabled build of ch (using the command-line option).  JsRuntimeAttributeEnableExperimentalFeatures doesn’t enable it so there’s no way to use the feature at all in a Release build.

I'm afraid my yield* implementation is wrong both here and in the further work I've done offline - it's right 95% of the time but gets edge cases wrong - struggling to come up with a way to handle it correctly and nicely other than the option of not doing the yield* loop in bytecode.
As far as I can see this would be:

much easier to implement the spec
probably more performant (as yield* would no longer involve yield/resume for each iteration)

BUT for error handling the stack trace if the iterator threw would not include the generator function which would not be ideal.

UPDATE: see edited first post for details - this is now a complete implementation of ES2018 Async Iteration (though could do with more tests)
Please could people find some time to begin reviewing this?

Awesome. Thanks, @rhuanjl . I'll give this a close review this week.

Thanks @rhuanjl and @pleath - it will be really awesome to get this in!

FYI, I'm about half way through the review, and so far everything looks nice and clean. I want to walk through a couple of use cases before I sign off.

This still is not usable with JsRuntimeAttributeEnableExperimentalFeatures, which is the only way to get access to experimental stuff in the release build - can that be fixed?

@pleath thanks for the prelim feedback - I've done a re-read/check and noticed a couple of things, not going to update the PR whilst you're mid review, notes for ref:

I realised that the JIT handlers I included in lower.cpp are untested as by default jilting generators is disabled (controlled by flag JitES6Generators) BUT - it's disabled as it doesn't work - jitting sync generators causes a crash - async ones crash the same way - so don't know if my code is ok and it's just the existing problem that's an issue or if my code is wrong.

Some of my spec text comments have the wrong numbering (oops, I'll fix this in a separate commit after you've reviewed)

to @fatcerberus point on JsRuntimeAttributeEnableExperimentalFeatures it seems this enables flags that use the macro FLAGPR_REGOVR_EXP() and not FLAGPR() - not sure which this should be

Not sure what to do for better test coverage - the basic cases I've included clearly aren't enough - test262 has ~2000 tests for this; it might take a long time to create full coverage.

@pleath Any update on this - as fixing the hit piece is out of scope per discussion in #5877 only other points I'm aware of are that it probably needs more tests and a couple of my comments are wrong - didn't want to add anything when you were mid-review though
(Also really expected your review to find some errors in my code this is the largest piece of c++ I've ever written)

(Hah. My previous comment was mangled, but I’ve fixed it.)

I've squashed my previous commits
Rebased to latest master and regen'd byte code (to fix merge conflict)
Updated the comment error + points from @pleath's review as the new commit "fixes"

Note this PR still adds this as a disabled feature - partly as no one has told me to enable it since I opened the PR and partly as I'm worried my test coverage is insufficient for a shipping feature this big.

Keep in mind that if the feature is fully disabled, including with the EnableExperimentalFeatures runtime attribute, then nobody can test it unless they have a flags-enabled build of CC.  That's not likely to be the case for embedders shipping an application, so the feature will get no real-world testing.

@fatcerberus I'm hoping this can be enabled soon but leaving that decision to the CC team - as an additional note node-chrakracore uses EnableExperimentalFeatures so if I switched it to an experimental feature of that kind it would be on in node-chakracore - it should really have TTD support before then I assume; and I don't know how to do TTD support.

I'm hoping this can be enabled soon but leaving that decision to the CC team

Yes, I was aware - I was simply bringing the above up as a discussion point.

Would it be possible to move forward with this or is it parked indefinitely?
Is there anything specific I can do to move this towards being mergeable?

Hi @rhuanjl - Thanks for pinging on this! I intend to pick this up within the next few days. In the meantime, would you mind doing a rebase?

@zenparsing I saw it got out of date, and had bytecode conflicts so I've just regenerated the bytecode and rebased again. I also squashed the fixes commit for neatness.
Please let me know if you'd like me to do anything else on this.

Thanks for merging this @zenparsing
As noted above whilst this is a full implementation it's disabled by default - please let me know if there's anything I contribute to help get towards it being enabled by default.


Support nordic characters

In one of my project http://shrgrp.org, the names of objects may contain some Finnish characters, like:
hyväntoivonkatu 6

On English keyboard, it is hard to type "ä" and I hope I can search it by typing "hyvantoivonkatu". Is it supported by js-search?

No, this isn't supported by default. You would need to implement your own IndexStrategy that mapped characters like "ä" to their more easily-typed variants (eg "a").
I don't have plans to implement this myself. If you write it in such a way as to be reusable for others though I would be happy to review a PR for consideration to be added into js-search 😄  Otherwise you can pass in your own implementation:
const search = new JsSearch.Search('isbn');
search.indexStrategy = YourIndexStrategy;


ballot_measure_selections to xml_tree_values

BallotMeasureSelection! Take 5.1 CSV data and transform it as if it were imported from XML.
Pivotal story: 118947297

LGTM, @tie-rack.


Replace includes with excludes in travis.yml

This appears to fix #809 for now.  I don't think it's ideal since the excludes are maybe harder to quickly process, but trying to make it use includes only doesn't appear to work right now.

Looks like you're going to need to cancel a few builds in order for the build for this pull request to run successfully.

Hold off on this one, I think I had a different issue messing with the includes (travis build queue was stuck), so I'm going to try that again.

Ok, I'll continue cancelling builds though :)

This one passed now.

Closed


Allow hardware information to be shown

Allow hardware information to be displayed (CPU, RAM, Flash), if it is present in the JSON data.
Rows are also hidden when there is no data.

@aparcar is that something nice to have? Not that we use this feature now, but it might become useful in the future.

Can you show a demo?

https://mwarning.de/yafs/www/
Search for "1&1 HomeServer"

But it might be nicer to show hardware data to the build data, not on top...

Beside the design decision, I think this gets quite big the more data you add also becomes less easy merge-able with the OpenWrt profiles.json file. Don't you think it is better to have a second file containing such data?

A second file might be good idea.

Anyway, this feature is not something we need right now. But it is interesting to discuss this.

Fine with me, although I did not see this in action yet.
There are some additional hardware information which could be valuable, some time ago I created a mock up here
The data is available as YAML, not JSON, but could be easily converted. If we'd stick to the YAML we could collaborate with the official OpenWrt wiki and have a community organized database.
A YAML parser for JavaScript exists here

This looks awesome. :-)
But it might be better to setup this in a different project. Both project scopes might be too divergent.

The code is now easy to extend to allow this. But this is out of the scope of this project. Closing.

Actually it could be interesting to load such information at some point. It's all available as YAML but could be easily converted to JSON https://github.com/aparcar/devices/blob/main/data/8devices_carambola.yaml


Partial stroke painting

Problem
The stroke of poly is only partially painted. See example below.
Input
julia> using GLMakie

julia> poly(Circle(Point(0.0, 0.0), 1), strokecolor=RGBAf(1.0, 0.0, 0.0, 1.0), strokewidth=5, color=RGBAf(0.0, 0.0, 0.0, 0.0))

Output

Expected
Circle as stroke.
Version
julia> versioninfo()
Julia Version 1.7.1
Commit ac5cc99908 (2021-12-22 19:35 UTC)
Platform Info:
  OS: Linux (x86_64-redhat-linux)
  CPU: AMD Ryzen 7 5800H with Radeon Graphics
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, znver3)
Environment:
  JULIA_NUM_THREADS = 16

GLMakie v0.4.7
Makie v0.15.3

Ugh, cant replicate this on master/tagged :(  So, I guess this is a duplicate of:
https://github.com/JuliaPlots/Makie.jl/issues/1460


QUESTION: there is a way to use uuid like ObjectId() in mongoDb

Sorry by this question but:
there is a way to use the uuid like ObjectId() in mongoDb
Regards.

If your question is; can I use a UUID as my primary key in a mongodb collection, then yes. It's as simple as saving your documents with { _id: uuid }.

this question would receive a better response from the mongodb team.


Screenshare audio for IOS sdk

Is your feature request related to a problem? Please describe.
Currently IOS app does not support the audio of shared screen.
Describe the solution you'd like
IOS app should support screenshare audio like web sdk has.
Describe alternatives you've considered
Additional context

Hi, unfortunately this is not supported at the moment .
I would definitely like to implement this.

This is related / duplicate of https://github.com/livekit/client-sdk-swift/issues/385
I will close this one.


How to implement Vue.js in this boilerplate?

Hello! I've just found your boilerplate, cloned the repo and installed dependencies. But now I can't understand how to implement Vue.js to this project. I want to develop front-end Vue and back-end Express simultaneously in one project with webpack. Please tell me how to do that

Vue allready installed, just import him in src/js/main.jj and done.

Thank you very much, I really appreciate your fast answer) I see your point


Dependency http2: new major version available

http2 is up to version 4.1.4, while the latest that twain can be built against is 3.0.3.  It would be helpful if twain could be migrated to the 4.1.x version http2 so that it can be used alongside other libraries that have http2 4.x as a transitive dependency.

http2 was bumped with <IP_ADDRESS>


updating links mmctl

The mmctl docs had two broken links. This is the fix.

Hello @JtheBAB,
Thanks for your pull request! A Core Committer will review your pull request soon. For code contributions, you can learn more about the review process here.


Localization API for plugin messages/strings

Per http://oxidemod.org/threads/multi-language-localization-api.7614/.
Progress has been made on this by a community member, it just needs to be reviewed and properly implemented into Oxide Core. See http://oxidemod.org/threads/multi-language-localization-api.7614/page-3#post-85941.

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/24616783-localization-api-for-plugin-messages-strings?utm_campaign=plugin&utm_content=tracker%2F6855523&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F6855523&utm_medium=issues&utm_source=github).

Implemented as of PR #598.


Provide more explicit instructions for AD/LDAP configuration

The instructions for setting up AD/LDAP only include this line for configuration AD/LDAP, but if you choose a non-static field for the IdAttribute you will end up deactivating users because this is both unique and case-sensitive.
This document should be modified to warn users of this problem..

hi @icelander - I wanted to find out whether you are open to providing the updated content so we can add it to the documentation. Thank you!


Humdrum editorials/ part splits cause Nans in duration features

Python: 3.12.4
Partitura: 1.5.0 (develop branch)
When loading BWV 875b:
import numpy as np
import pandas as pd
import partitura as pt

score = pt.load_kern('testdata/875b.krn')

note_array = score.note_array(
    include_pitch_spelling=True,
    include_metrical_position=True
)
df = pd.DataFrame(note_array)
df[df.isna().any(axis=1)]

duration_beat and duration_quarter are NaN for

During import I also get warnings:
partitura/io/importkern.py:277: UserWarning: Part P2 already exists. Adding to previous Part.
partitura/io/importkern.py:277: UserWarning: Part P1 already exists. Adding to previous Part.

The first NaN seems to be caused by the editorials in measure 9
=8	=8	=8
*	*strophe	*
*	*^	*
*	*S/sic	*S/ossia	*
16DL	16f#xxL	8f#xx	8cc]L
16d	16dxxJ	.	.
*	*v	*v	*
*	*Xstrophe	*
16e-	4g	8b-J

Thanks Gregory for your issue and for reporting those errors, I am on it!! I will work on a solution ASAP.

Update on the issue
The editorial comments where not supported previously and as a result they could cause error during parsing. We will follow a similar approach to Verovio Humdrum Viewer where only the original version is kept and alternative passages are discarded. The PR #370 addresses such issues, it will be merged soon to the develop branch and will be included in main with the latest release. If you encounter other error please reach out!


Can't create container on new version

After I upgrade to newest version of weave, can't create any container after I execute
eval $(weave env), it will froze.
I wait long time and I have to use Ctrl + c to stop the process and get this error
Error response from daemon: invalid CIDR address: Allocate
If I run weave launch and don't run  eval $(weave env), I can create container as normal

After I upgrade to newest version of weave, can't create any container after I execute
eval $(weave env), it will froze.
I wait long time and I have to use Ctrl + c to stop the process and get this error
Error response from daemon: invalid CIDR address: Allocate
If I run weave launch and don't run eval $(weave env), I can create container as normal

I suspect your weave network is lacking IPAM consensus, presumably because some peers are not connected. Check weave status.
This is not a new behaviour in 1.1.

Thank you. I fixed it.
I am upgrading old weave to new version, that caused this issue, if there's peer connected to the new weave server used older version of weave, this will happen.
After I start all the new sever, all working.
Thanks

if there's peer connected to the new weave server used older version of weave, this will happen.

Right. See #1419.


403 HTTP error on co win API

HI @vic7z I can work on this. Can you please assign this one to me under hacktoberfest tag?

Yeah sure but this project is long dead and i don't even know the API still works but i will assign you the issue

HI @vic7z working on this


Adding missing enzymes from EC to GO

Ron Caspi<EMAIL_ADDRESS>has done an analysis and finds   2188 EC terms are missing from the EC2GO mapping file. This number includes some classes and subclasses, e.g. EC-5.99. There are 2111 complete numbers describing specific enzymes.
I have attached two excel files with data, one with classes, the other with only enzymes (4 digit ec)
Complete definitions of the EC numbers can be found at enzyme-database.org
http://www.enzyme-database.org/query.php?ec=<IP_ADDRESS>
That website also has the definitions of the classes and subclasses - you can find them manually under the "Enzymes by Class" tab, or by using urls in the format
http://www.enzyme-database.org/cinfo.php?c=X&sc=Y&ssc=Z   (c=class, sc=subclass, ssc = subsubclass).
For example http://www.enzyme-database.org/cinfo.php?c=4&sc=5
This will be a large project.
EC numbers not in GO with classes.xlsx
EC numbers not in GO no classes.xlsx

Chris might have made a script that used KEGG

Are we still planning to do this ?

This is ongoing as some are added during other tickets. And we do plan that once we are all set with RHEA we can fetch the ECs for reactions from RHEA.

If I understand correctly, this is covered by other tickets. Can we close?

Not really covered in other tickets. This is a different approach. I haven't been adding new terms from this list directly. It's an old list now. What would be useful is to get a recent ist of  4 digit ecs that are NOT in GO. But to use them to add, I would either need a rhea to use as a definition ref, and/or a PMID

Do we want to add a reaction unless a specific request for it comes in?

Classic answer - no; wait for term requests and then create as needed.
But it's probably time to re-visit this in co-ordination with Rhea and, here, Ron Caspi. I get the impression that Rhea's preferred practice may be to take on a domain of chemistry and handle it exhaustively (but Alan Bridge would need to speak to this) and if that fits with MetaCyc curation plans that are fairly concrete, then maybe the requested bulk term creation could make sense? That would also make it much easier to ensure that all the new terms are consistent and consistently grouped - easier than for piecemeal term creation? @ukemi ?

I just imagine the amount of time going down a list of 4 digit ECs not in GO. It could be done bulk computationally if the EC already had a single rhea OR at least a PMID.

My two cents is that we need to at least coordinate with Rhea based on our recent strategies to generate and exhaustive 1:1 mapping.

We'll revisit this when the Rhea first load is done.

Greetings. NCBI's prokaryotic genome pipeline group has been adding GO terms to annotation rules used by the PGAP and RefSeq pipelines, and fairly soon will let those annotation rules attach GO terms to a substantial fraction of the ~200,000,000 different prokaryotic proteins in our collection.
We are interested in obtaining the latest EC2GO mapping to assist us in this.  If the list is less up to date than it could be, we would like to join the set of voices asking for a more complete EC2GO.

Hi @danielhhaft
The most recent EC2GO and RHEA to GO mappings can be found at this address:
http://current.geneontology.org/ontology/external2go/index.html
They are updated approximately monthly, at each GO release.
Thanks, Pascale

@pgaudet  is this out of date?

@pgaudet  this seems out of date based on the Rhea alignment, should we close? If anyone needs the terms that are in EC  but not in GO for annotation they would request.
v

Right, we should run this again at some point.


Make RPC to clear ping counters

Closing this for the moment as it hasn't been a requirement lately, we can discuss it again if we have a case...


KNL-1590 - Add session.parameter.allow.bypass

Make ContentItem work without requiring session.parameter.allow
by adding a finer grain regex based on the URI.
session.parameter.allow.bypass=sakai.basiclti.admin.helper.helper

I think this is a good idea, and was thinking this might be needed to allow some tools to exempt themselves. I don't know if we should have this parameter as a regex though, seems like a csv getStrings would make it easier to use. I don't think anyone is going to do a wildcard, and you can just turn the SameSite off entirely. This will make a little easier to set the value of the property.

Yeah, java just annoys me since it needs the double slash for escaping regex. ;)
We could probably put up an init-param on the RequestFilter. I'm just skeptical if we'll find many classes we'll have to exempt from this that we'll actually have to configure this property.
Also if the regex stays it should probably catch the PatternSyntaxException with a log error. I'm guessing it would result in a stack trace otherwise and we really don't want to stacktrace out an init method and have a mis-configured property possibly stop startup.

Hey @jonespm I talked to @ern about this and went over why this needs to be global and a regex.  I pushed the try / catch change.  Good thing - as you surmised - a bad regex stops Sakai from starting.  It is not pretty.  But now you just get warned about about syntax errors.

Hey @ern - the code is already in the init() method.

I am going to merge this. The log message format is the only comment not handled and the log message format is the same as for the rest of the module.  I will leave the log message approach for later.


model Salesforce/xgen-7b-8k-inst

Hello. I'm trying to use the new model of Salesforce
Salesforce/xgen-7b-8k-inst
SELECT pgml.transform(
    task => '{
      "task": "text-generation",
      "model": "Salesforce/xgen-7b-8k-inst"
    }'::JSONB,
    inputs => ARRAY[
        'Once upon a time,'
    ],
    args => '{"max_new_tokens": 32}'::JSONB
);

it gives the error:
ERROR:  Traceback (most recent call last):
  File "", line 141, in transform
  File "/var/lib/postgresql/.local/lib/python3.8/site-packages/transformers/pipelines/__init__.py", line 885, in pipeline
    tokenizer = AutoTokenizer.from_pretrained(
  File "/var/lib/postgresql/.local/lib/python3.8/site-packages/transformers/models/auto/tokenization_auto.py", line 688, in from_pretrained
    raise ValueError(
 ValueError: Tokenizer class XgenTokenizer does not exist or is not currently imported.

SQL state: XX000

The XgenTokenizer is in the repository:
https://huggingface.co/Salesforce/xgen-7b-8k-base/blob/main/tokenization_xgen.py
Is possible to import somehow this .py file to solve the error?
Thanks

Looks like this model needs an additional dependency from https://huggingface.co/Salesforce/xgen-7b-8k-base:
pip install tiktoken

Maybe we should add this to the requirements.txt.

Yes, I had already installed tiktoken before sending the error here, so the problem seems to be centered on the inability to load the XgenTokenizer class. Is there a way to do it?

Looks like they are using some config to replace the standard AutoTokenizer, although I'm not sure how that config is typically executed by HF.  The example in the model card seems like it should "just work" like all the other auto_tokenizers, but there may be some missing additional permissions or config for PostgresML.

I made it work! Adding "trust_remote_code": true
SELECT pgml.transform(
    task => '{
      "task": "text-generation",
      "model": "Salesforce/xgen-7b-8k-inst",
	"trust_remote_code": true
    }'::JSONB,
    inputs => ARRAY[
        'Once upon a time,'
    ],
    args => '{"max_new_tokens": 32}'::JSONB
);


Current mincer dependency references the deprecated autoprefixer-core

Hi,
As the title says, the current<EMAIL_ADDRESS>dependency references deprecated autoprefixer-core. This results in project using connect-assets with autoprefixer functionality having to list the deprecated library as one of the dependencies.
The latest<EMAIL_ADDRESS>now references autoprefixer (the current official repo & module name). Please see related mincer changelog for more information.
I already tried upgrading and fixing the 8 failed tests the upgrade causes to no avail :sob: I think it'd better be put in good hands.
Thanks!

Any updates?

Any updates on this? Mincer is now at 1.4.1.

Just published v5.2.0, which now depends on Mincer 1.4.1.


Header issues on individual pages

The light/dark mode toggle button is absent on individual project and blog pages, and on the profile pages it is present but the mobile navigation icon is overlapping it:

I believe users should be able to toggle between light and dark mode even on individual pages, so I suggest making sure the toggle button is present on blog, profile, and project pages. Additionally, the mobile nav icon should not be present on these pages since there is no navbar needed on individual pages.

@marcnjaramillo Works fine for me on Chrome. Which browser did you observe this on? Need to look into why we are observing different behaviour

@marcnjaramillo
!image Works fine for me on Chrome. Which browser did you observe this on? Need to look into why we are observing different behaviour
image
image

Great! I've assigned it to you @RishiDesai17


A potential approach to enhance idea generation.

Nice work!  I'm impressed by the automatically generated paper!
I’m working on a project called “AutoSurvey” which aims to generate comprehensive literature surveys. I think it complements what you’re doing quite nicely—while scientific discovery focuses on scientific discovery, we’re tackling the summarization and review of existing research.
Based on my experience in AutoSurvey , I would like to suggest a possible approach to help idea generation. According to the process described in your paper, AI-scientist first brainstorms ideas based on its own knowledge and then refines the ideas iteratively. However, a general research paradigm typically involves extensive reading of existing literature to discover new ideas. It seems that the latter approach (such as using RAG techniques to hierarchically reading hundreds of papers) could better utilize LLMs' capability to process long texts.
Have you ever considered this approach?

Thank you! That work is very interesting, and we're adding it to our related work section. I think that would be a great idea to incorporate an extensive literature survey, and I wonder how the ideas generated would compare. Please do feel free to propose a change to our generate_ideas file! :)

keep in mind the approach suggested by guoqi2000 (first searching the literature and then bruteforce generating novel ideas) is computationally more expensive than the current approach (first discussing internally then looking up the literature to check whether the initial idea is novel)


chore: bootstrap releases for path: .

Configuring release-please for path: .

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 100.0%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
6627

Relevant Lines:
6627

💛  - Coveralls


Add ability to fetch data from Spaces

With some additional work, it would be possible to support fetching data from Spaces and Spaces recordings, which would allow to cover more surface area from the API.
Looking at the network tab when navigating through Spaces recordings, it seems the data is fetched similarly to Tweet details. With minimal adjustments, we should be able to support the feature.

I never knew about spaces. Where can I access it from the web app?

 If possible, I'd like to work on it.

After a second look, maybe this issue fits best in the rettiwt-core repo.

@rolivencia Yes indeed. However, these changes will eventually arrive here, although it will be a while before they do.

Hi any updates on retrieving spaces?

Nope, I have not yet started working on spaces.


Scenario updates, incorporating scenarios from @JustinCappos

Incorporates feedback from #12
Signed-off-by: Steve Lasker<EMAIL_ADDRESS>

Can we get some :eyes: on this so we can close these out?

@JustinCappos, can you ack that we've captured the scenarios portion

The document is really design heavy now. This covers a few cases but things like 7 only make sense in the context of a specific design, whereas 8 and 9 are much more general.

Are you referring to the scenario or implications? Do you have a proposed edit so we can merge?

The document is really design heavy now. This covers a few cases but things like 7 only make sense in the context of a specific design, whereas 8 and 9 are much more general.

Are you referring to the scenario or implications? Do you have a proposed edit so we can merge?

It's more of a conceptual change to parts of the document.  Some of the content like this feels like it is quite specific to the technologies and not the goals / scenarios.  I'm just going to cut and paste a few example, but the conceptual difference is something I'm seeing throughout.
"The ACME Rockets environment enforces various company policies prior to any deployment, evaluating the content in the SBoM. The policy manager trusts the content within the SBoM is accurate, because they trust artifacts signed by wabbit-networks. The src content isn't evaluated at deployment time and can be left within the registry.

Once the policy manager completes its validation, the deployment to the hosting environment is initiated. The SBoM is no longer needed, allowing the image to be deployed separately. A deploy artifact, referencing a specific configuration definition, may also be signed and saved, providing a historical record of what was deployed. The hosting environment also validates content is signed by trusted entities."

"The local environment has a policy by which it states the set of keys it accepts.

The signing and validation of artifacts does not require a registry. The local host can validate the signature using the public keys it accepts.
The key used for validation may be hosted in a registry, or other accessible location.
The lack of a registry name does not infer docker.io as a default registry."

"Users may reference the sha256 digest directly, or the artifact:tag. While tag locking is not part of the [OCI Distribution Spec][oci-distribution], various registries support this capability, allowing users to reference human readable tags, as opposed to long digests. Either reference is supported with Notary v2, however it's the unique manifest that is signed."
I don't really know if this should be moved to a separate document, if general text should replace it, or if I'm just keying in on things that I shouldn't.
Perhaps @justincormack should weigh in also and say whether what I'm objecting to makes any sense.

I don't really know if this should be moved to a separate document, if general text should replace it, or if I'm just keying in on things that I shouldn't.

@JustinCappos I think we're possibly seeing a view from different perspectives. There's a perspective that we need a signature solution to artifacts in a registry. A solution that can move with the artifact as it moves within and across registries. That does define a sandbox by which we're working within. In the call last week we discussed how we may need to tweak the sandbox, but we are scoping to storing signatures in a registry.
The work you've done in Tuf is about taking validation to a another confidence level. We've been talking about scope creep quite a bit, and how we can minimize that scope creep, working from a base that covers the scenarios and building upon it. For instance the generic SBoM references may refer to a Tuf implementation, a RedHat implementation or others.
Now I know @justincormack was thinking it might be more broad, and he was writing up something. This seems like the conversation we should be having. What is the scope of what we want to carve out here.


JUnit5 Migration

What better way to figure out exactly how hard migrating to JUnit5 is than doing the whole migration? That's what I ended up doing.
I haven't had time to thoroughly go through every single test to guarantee everything is working. But all the tests pass, I've tried my best to be careful in rewriting things. The test count ends up slightly different between JUnit4 and JUnit5 for reasons I've not fully accounted for, but I haven't seen any tests being dropped.
I have not fundamentally rewritten any tests that didn't map more or less directly to JUnit5. I've migrated the ones that had a clear mapping.
I'll explain all the pain points, incompatibilities and other relevant things.
I don't expect this to be merged since it's such a huge PR and I can't ensure prompt availability if it subtly messes something up, but it is fully functioning as far as I can tell, so it's not completely out of the question, I suppose. The purpose was more to get a good idea of the process.
How
The migration was done by a combination of project-wide string replacing JUnit4 imports and annotations with JUnit5 imports and annotations then manually combing all the files for any needed local edits.
General Considerations
It's important to note that JUnit5 and JUnit4 tests are perfectly capable of co-existing within the same project.
The golden rule I encountered was to not have any JUnit4 tests interact with JUnit5 tests, e.g. by inheritance. JUnit5 has a Vintage runner and supplies all of the JUnit4 dependencies so the old imports still work. This means that the migration can be done incrementally.
The way it works is that any annotations sourced from JUnit4 imports will run on the Vintage runner automatically and annotations sourced from JUnit5 imports will run on the new runner.
JUnit5 requires Java 8 at runtime but can test code compiled with Java 7 and runs perfectly fine in language level 7 modules as long as you don't write language level 7 code.
In practice, this is only mildly annoying due to lambdas and assertThrows, which I'll mention in detail later. Otherwise, it all seems to work perfectly fine.
Migration
1. Pom Files
The easiest way to introduce the new JUnit5 dependencies is to replace the JUnit4 dependencies with these ones:
<dependency>
    <groupId>org.junit.platform</groupId>
    <artifactId>junit-platform-launcher</artifactId>
    <version>1.4.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-engine</artifactId>
    <version>5.4.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.vintage</groupId>
    <artifactId>junit-vintage-engine</artifactId>
    <version>5.4.2</version>
    <scope>test</scope>
</dependency>
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-params</artifactId>
    <version>5.4.2</version>
    <scope>test</scope>
</dependency>

It's possible to stop right here and the project still works and runs tests properly.
2. Before/After
These annotations are pretty simple to migrate from JUnit4 to JUnit5 and present no difficulties:
BeforeClass -> BeforeAll
AfterClass  -> AfterAll
Before      -> BeforeEach
After       -> AfterEach

org.junit.BeforeClass -> org.junit.jupiter.api.BeforeAll
org.junit.AfterClass  -> org.junit.jupiter.api.AfterAll
org.junit.Before      -> org.junit.jupiter.api.BeforeEach
org.junit.After       -> org.junit.jupiter.api.AfterEach

There's one thing to keep in mind: The new JUnit runner will throw an exception if methods annotated with Before or After are declared static. In every place I've encountered, the static keyword could be safely removed and things continued to work.
3. Assert/Assume
JUnit5 has perfectly equivalent asserts and assumptions. The import has changed but the method names have not.
org.junit.Assert -> org.junit.jupiter.api.Assumptions
org.junit.Assume -> org.junit.jupiter.api.Assertions

There's one hugely annoying issue though. This is probably the biggest time waster in the migration. JUnit4 asserts have a parameter list that looks like assert(condition, message) whereas JUnit5 has assert(message, condition). This means you can easily waste an hour or two just switching the parameters around. To my shame, that's exactly what I resorted to. Someone more capable in text munging could write some script to do it automatically in a way that doesn't hit edge cases. Besides this, there aren't any other concerns.
4. Theories
Theories are completely unsupported. They should be ran as JUnit4 tests.
JUnit5 has decided to not add them, so the concept is deprecated for all intents and purposes. This comment recommends using property-based testing in a third party runner such as jqwik.
My recommendation is to stop writing theory based unit tests. If datapoint combination functionality is required, then use something like CartesianProductProvider.
5. The New assertThrows
Both these forms of exception throwing tested are deprecated in JUnit5:
@Rule
public ExpectedException thrown = ExpectedException.none();

@Test(expected = IllegalArgumentException.class)

The new way to deal with this is assertThrows. The only problem is that it takes a lambda as one of its parameters and a lot of the modules are still on language level 7 which doesn't have lambdas.
@Test
public void testEncoderWithWrongVersionIsRejected() {
    assertThrows(IllegalArgumentException.class,
        () -> EncodingManager.create("car|version=0"));
}

However, there's an easy workaround. Its only downside is that it's somewhat ugly.
@Test
public void testEncoderWithWrongVersionIsRejected() {
    assertThrows(IllegalArgumentException.class, new Executable() {
        @Override
        public void execute() throws Throwable {
            EncodingManager manager = EncodingManager.create("car|version=0");
        }
    });
}

It's perfectly functional and offers a clear way of migrating this feature from JUnit4 to JUnit5.
6. Rules
Rules are entirely unsupported in JUnit5. There is some life support for a very limited set of rules using the migration module. The closest equivalent JUnit5 has is Extensions.
Conclusion:

Stop writing new tests with Rules when possible.
Keep existing tests on JUnit4.
Use callbacks or Extensions to get similar functionality in new tests.

7. Parameterized
There is no way to properly emulate JUnit4 Parameterized in JUnit5.
JUnit4 achieves test class-wide parameterization by using a static source to get parameters and using them on the constructor of the class.
JUnit5 only does per-method parameterization.
You can approximate some of what JUnit4 does with BeforeAll + JUnit5 ParameterizedTest but the rewrite ends up being pretty extensive. JUnit5 extensions might be able to fill this gap, but I haven't figured out a clear way of doing so yet.
There are discussions of having this feature in JUnit5. Nothing concrete yet, but it's likely that @ParameterizedTest will be changed to work on classes as well as methods when 5.5 ships.
Instead of trying to approximate per-class parameterization in JUnit5, an option is to rewrite existing tests to use per-method parameterization. This is perfectly possible but somewhat time consuming and the other downside is that it more or less forces the setup to be done for every test method in part. This is not a problem in some places, but might make tests a lot slower in others. There are laziness schemes that can allow the setup to run only once and then be 'cached' for the rest of the tests if that's too much of a problem.
Conclusion:

Stop using Parameterized and RunsWith when possible.
Write new tests with per method parameterization in mind.
Keep old Parameterized tests on JUnit4 (Vintage runner).

8. Ignore
Ignore has been replaced with Disabled.
org.junit.Ignore -> org.junit.jupiter.api.Disabled

9. Summary of the major pain points

Class-wide parameterization cannot be done in JUnit5.
RunsWith superseded by ExtendWith.
Entire Rule system deprecated in favor of Extensions.
New Assert/Assume methods reverse message and condition parameters.
Thoeries have been completely deprecated.

Performance
I did some very basic benchmarking and it seems that JUnit5 is slightly slower.
JUnit4
Total time:  53.497 s
Total time:  54.360 s
Total time:  51.140 s

JUnit5, no tests migrated to the new runner
Total time:  59.478 s
Total time:  01:04 min
Total time:  57.331 s

JUnit5, most tests migrated to the new runner
Total time:  01:05 min
Total time:  01:00 min
Total time:  59.978 s

mvn install was run once for each category and then mvn test was run three times. The three mvn test total times were noted for each category.
Please note that the testing conditions were far from ideal, with many things that could bias the tests, including variable background CPU load. This is just meant to give a very rough sketch.
It's worth taking into consideration that JUnit5 has support for running the tests in parallel that was not enabled as it requires some effort to set up. But that's a future avenue for performance improvement.
Test Count
Are all the tests there? We didn't lose any along the way, right?
feature/junit5-migration --> Tests ignored: 12, passed: 2104
master                   --> Tests ignored: 8, passed: 1914

This was startling but upon further investigation the stark difference is largely the result of JUnit5 counting repeated tests and JUnit4 not doing so.
I counted every test repetition: 100 + 10 * 11. That's 210. 2104 - 210 = 1894. That's still a difference of 20 tests. I combed manually through the test runner and couldn't see any problem. I tried to output mvn test to a file with some grepping and diff the JUnit4 and JUnit5 outputs, but I ran into problems with outputs being ordered differently.
The ignored tests are the same when looking through IntelliJ's test runner:

JUnit4

JUnit5
So it's most likely just counting them differently.
Overall Recommendation
JUnit5's attractive features are:

Per method parameterization that can make some tests look so so much better.
Better assertion-based way of testing for thrown exceptions.
Ability to use nested test classes without any fuss (e.g. without RunWith Enclosed).
The Tag annotation allows an easier and more structured way to filter out lengthy tests.
Can programmatically generated new test cases using the Dynamic Tests feature.
Native support for repeated tests via the @RepeatedTest annotation.
Parallel Execution is still experimental but easier to use than JUnit4's ParallelComputer.

I'm not sure what these mean to the GraphHopper team overall, if any of these are more than merely "nice to have".
Migration is pretty easy overall, it took me basically just an entire day by myself.
Going to JUnit5 is definitely more future proof. But that depends what you're proofing against. JUnit5 will continue to be worked on while JUnit4 will be on bug fix support mostly. The last release for JUnit4 was on 2018-11-25.
One thing is for sure, continuing to write JUnit4 tests will make ripping the bandaid off at least somewhat harder later. New tests written to use Theories, Parameterized, RunWith or any Rule will make migration increasingly more difficult.
If JUnit5 ships class-wide parameterization in 5.5 then Parameterized tests have a clear migration path.
However as the size of the codebase grows, migration will mean touching a higher amount of tests and their code, so more places to mess up when migrating. So a very minor form of technical debt would still accrue regardless.
I would recommend at least switching to a JUnit5 dependency and writing new tests to be run by the JUnit5 runner while letting old tests run on the Vintage runner. The only cost of doing that is slightly slower execution speed, at least without parallelism.
Extensively rewriting old tests that use Parameterized or Rule is very likely not worth it at the moment.
The tests with clear migration paths have very good time investment ratios. I know, since I've basically already migrated all of those.

If, when and how to migrate is up to GraphHopper. I hope this is at least informative.

Thanks a lot for this effort :)
It is interesting as it comes not only with advantages - like with so many decisions ;)
But it is likely that we need to migrate to junit5, maybe not now but as soon as possible. Will have to think about it. Maybe others can jump in with their opinion about it :)

Thanks a lot @Anvoker, I can tell you put in a lot of work here and for sure it helps a lot making a decision about the migration! I will have to look in detail and next time I am working on some tests I can think about what would have been different with JUnit5. Being able to do a soft migration (migrate those test that are easy to migrate) first and do it step by step (maybe whenever one is working on a test) is a plus in my opinion.

Closing in favor of #https://github.com/graphhopper/graphhopper/pull/2324


Other distributions for IBP init

Hi,
Have you tried other distributions for IBP init? Does it make difference?
Since the IBP training has non-continuity at W_i = 0, would it make sense to initialize it with Laplace distribution W_i ~ Laplace(0, b)? Then |W_i| ~ Exponential(b^{-1}), so b = 2/n_i would also work.
Best regards,
Zhakshylyk

Hi Zhakshylyk,
I think we just assumed normal distribution. Sounds interesting to consider other distributions. Could you please explain a bit why making the distribution of W_i non-continuous at 0 would potentially help the training?
Thanks!
Zhouxing


wiki issue: projectFlavor explanation

On the docs (https://github.com/typesafehub/sbteclipse/wiki/Using-sbteclipse), under the projectFlavor section, the example appears to be a copy-paste of the example from the previous section (executionEnvironment)

projectFlavor
This setting of type EclipseProjectFlavor.Value, which defaults to Scala, is used to specify which builder and natures Eclipse should use. The available values are the Value members of the EclipseProjectFlavor enumeration: Scala, Java.

The following example shows how to use the execution environment for Java 6:

EclipseKeys.executionEnvironment := Some(EclipseExecutionEnvironment.JavaSE16)

This is fixed already in the wiki. Can this bug be closed?


In-memory decryption?

As e.g. this HN comment points out, as well as this code, Ylva seems to decrypt the "vault" onto disk to read from it.
This feels like a fairly insecure design – would it not be better to decrypt the data from disk on-demand into memory in StorageManager (and similarly only write encrypted data when it's time to Preserve() things?

Hi,
Actually I'm working on this at the moment. There will be some other security related changes too. :)

Now by default, Ylva does not keep decrypted data on disk. There's an option available to keep the decrypted data on disk, but by default it's off.


Update SDL_gamecontrollerdb.h

Added MAYFLASH Arcade Fightstick F300 and Speedlink Competition Pro Extra joystick (AmigaInput recognize it as Game Controller for Android v2.0)

Merged, thanks.


Azure AD ambiguous

[Enter feedback here]
Select app in azure AD.. this is ambiguous, there are many azure AD entries in the portal.  Please update /test these instructions with latest state of Azure

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2cc0b77a-df21-7007-5cec-4399d98c6dc6
Version Independent ID: efe5ae5e-ff64-f1c7-6920-4512fc8f36d1
Content: Azure Key Vault Configuration Provider in ASP.NET Core
Content Source: aspnetcore/security/key-vault-configuration.md
Product: aspnet-core
Technology: aspnetcore-security
GitHub Login: @Rick-Anderson
Microsoft Alias: riande

there are many azure AD entries in the portal

There's only one entry per app per tenant in the UI. The instruction is just saying to select the app that you intend to work with and then navigate to Certificates & secrets.

Thanks for contacting us. We believe that the question you've raised have been answered. If you still feel a need to continue the discussion, feel free to reopen it and add your comments.


feat: optimize segment proof aggregation

Resolves #399
Resolves https://github.com/0xPolygonZero/zk_evm/issues/403

PR limits the number of the segment proofs that are proven in parallel to limit the memory consumption with the small penalty of async engine utilization. Chunk size could be tweaked if needed.

At the moment, generate_next_segment is executed on GENERATION_INPUTS_AGG_CHUNK_SIZE_INNER GenerationInputs. 4 such jobs (so 16 SegmentProof proving) would run in parallel, and then be aggregated to chunk transaction proof.
Eventually all the chunk resulting transaction proofs would be aggregated to final transaction proof. Chunk sizes are there to limit memory consumption (we will keep in memory input GenerationSegmentData for GENERATION_INPUTS_AGG_CHUNK_SIZE_INNER * GENERATION_INPUTS_AGG_CHUNK_SIZE SegmentProof jobs). If that is not what we want, what is the alternative?
Original implementation was keeping all the GenerationSegmentData and executed first all the SegmentProof jobs before it switch to the proof aggregation (e.g.if block has  200 GenerationInpus then one data_iterator and SegmentProof for all segments in the block running in parallel), because paladin does not switch to other operation in chained operation setup before it finishes the previous one.

if block has 200 GenerationInpus then one data_iterator and SegmentProof for all segments in the block running in parallel

Put it differently, think that we now have a single GenerationInputs representing 200 txns. The issue is the internal SegmentDataIterator. There is only of them, but it is responsible for generating all segments, which cannot be processed as fast as they are being generated.
We still have the outer loop iterating over txns because we want the flexibility here. This is configured by the -b batch size argument when calling the proving mode. But theoretically, the best is when the batch covers a whole block at once, effectively removing the outer loop.

Returned it to draft, CI tests seem pretty slow (maybe there is some other reason), but I need to test this PR more with the real prover tomorrow.

PR is tested. Currently we have fixed parameters for cpu len and batch size of 10, where premature segment chunk aggregation does not make a big improvement in memory used, but takes significant CPU performance hit. We give up this approach.
Alternative that would be better but would probably take much more effort (if possible) would be to change Paladin to be able to run chained operations (map/fold) without waiting for the first operation to finish from all the workers.


Update version in README.md demo composer.json

The current version is very outdated.
Beginners use it without thinking and end up with cryptic errors :-(

:+1:

Thanks very much @oadam :-)


[Services] - Not able to create instances using docker-compose.yml when image points to private registry

Server version - V0.16.3
Steps to reproduce the problem:
Create a private registry - quay.io with valid credentials.
Create environment.
Create service using a private image in  this private registry (that requires credentials) or a public image in this registry.
In both the cases , I am not able to use the generated docker-compose.yml to create the instances using docker-compose command.
contents of docker-compose.yml
tregistry1:
tty: true
image: quay.io/sangeetha/test2
privileged: false
stdin_open: true
Sangeethas-MacBook-Pro:~ sangeethahariharan1$ docker-compose up ( test1 is private registry)
Creating sangeethahariharan1_tregistry1_1...
Pulling image quay.io/sangeetha/test1:latest...
v1 ping attempt failed with error: Get https://quay.io/v1/_ping: dial tcp: i/o timeout. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add --insecure-registry quay.io to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/quay.io/ca.crt
Sangeethas-MacBook-Pro:~ sangeethahariharan1$ docker-compose up( test2 is public registry)
Creating sangeethahariharan1_tregistry1_1...
Pulling image quay.io/sangeetha/test2:latest...
v1 ping attempt failed with error: Get https://quay.io/v1/_ping: dial tcp: i/o timeout. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add --insecure-registry quay.io to the daemon's arguments. In the case of HTTPS, if you have access to the registry's CA certificate, no need for the flag; simply place the CA certificate at /etc/docker/certs.d/quay.io/ca.crt
Sangeethas-MacBook-Pro:~ sangeethahariharan1$

Sangeetha, can't reproduce it on my setup. Here is the docker-compose.yml:
alena:
restart: 'no'
tty: true
image: quay.io/sangeetha/test2
privileged: false
stdin_open: true
And I am able to launch instances of this service. Can you follow the instructions in "If this private registry supports only HTTP or HTTPS with an unknown CA certificate..." and see if it fixes the problem?