Add a flavor of training_step that allows for expressing inter-batch parallelism

🚀 Feature

Motivation
When training a model, the most common approach is to process input batches sequentially (Lightning does this today). This makes sense, because in order to achieve synchronous/deterministic training, the processing of batch[n+1] can’t start until the optimization for batch[n] completes. However, this is only partly true, because it only applies to model weight-dependent logic. We can execute the model weight-independent logic out-of-order while still maintaining numerical equivalence. If the model weight-independent logic takes significant amount of wall time and stresses different system resources compared to the rest of the training step, parallelization can lead to increased throughput.
A trivial example of this is hiding the latency of host-to-device copy of input batches behind computational intensive operations. Without parallization, the execution looks like:
batch 0: [HtoD][forward][backward]
batch 1:                          [HtoD][forward][backward]
batch 2:                                                   [HtoD][forward][backward]

With parallelization, the latency of HtoD copy can be hidden:
batch 0: [HtoD][forward][backward]
batch 1:       [HtoD]             [forward][backward]
batch 2:             [HtoD]                          [forward][backward]

A more concrete use case for inter-batch parallelism at FB is to accelerate the model parallel technique used in DLRM (https://github.com/facebookresearch/dlrm). DLRM models consist large amount of embedding layers which are sharded across different GPUs. The forward pass for the embedding layers is roughly as follows:

All ranks receive global indices
The indices are routed to GPUs that hold the corresponding embedding weights (with alltoall collective)
All ranks perform local embedding lookup
The embedding lookup results are routed back to senders of the indices (with alltoall collective)

As we can see, step 1 and 2 are (1) model weight-independent (2) expensive (3) stresses communication resources. With inter-batch parallelism, we can hide the latency behind the previous batch’s backward pass.
Pitch
Support a training_step flavor with access to the data iterator, so users can express the aforementioned algorithms.
Alternatives

Additional context

@yifuwang how much do you want to tie this feature support to the LightningModule decomposition proposed elsewhere?
If we want to decouple this, in the short term we could follow a similar strategy that manual optimization took:

Introduce a new property and setter on the LightningModule e.g. automatic_iteration
Depending on the property set on the LightningModule, add conditional logic in the loops to dynamically send the right arguments to the step function (e.g. the batch or the data iterator)

Then in the long term, we continue formalizing these as separate interfaces
My assumption is that this would hold for all of training_step, validation_step, test_step, predict_step as the non-training ones would still benefit from the HtoD overlap with forward right?

Question, are you planning to let training_step take an union of a "dataframe" and an "iter of dataframe" or this will be a different function altogether?

@ananthsub I haven't considered how to approach this from LightningModule decomposition's perspective. But I think it makes perfect sense to do so.

Dear @yifuwang,
Would you mind pointing us to the current implementation of such features within DLRM library or provide a pseudo-code on how this would work ?
Best,
T.C

Hi @tchaton,
Sorry for the delayed response.
It looks like the DRLM project didn't include the optimization. However, a WIP recsys project that FB may open source soon will rely on the optimization. Please see below for a simulation of the problem:
Best,
Yifu
#!/usr/bin/env python3
import torch
from statistics import mean
import time

CYCLES_PER_MS = None

def get_cycles_per_ms() -> float:
    """
    Measure and return approximate number of cycles per millisecond for torch.cuda._sleep

    Copied from: github.com/pytorch/pytorch/blob/master/test/test_cuda.py
    """

    def measure() -> float:
        start = torch.cuda.Event(enable_timing=True)
        end = torch.cuda.Event(enable_timing=True)
        start.record()
        torch.cuda._sleep(1000000)
        end.record()
        end.synchronize()
        cycles_per_ms = 1000000 / start.elapsed_time(end)
        return cycles_per_ms

    # Get 10 values and remove the 2 max and 2 min and return the avg.
    # This is to avoid system disturbance that skew the results, e.g.
    # the very first cuda call likely does a bunch of init, which takes
    # much longer than subsequent calls.
    #
    # Tested on both Tesla V100, Quadro GP100, Titan RTX, RTX 3090 GPUs
    # and seems to return stable values. Therefore, we enable caching
    # using lru_cache decorator above.
    num = 10
    vals = []
    for _ in range(num):
        vals.append(measure())
    vals = sorted(vals)
    return mean(vals[2 : num - 2])

CYCLES_PER_MS = int(get_cycles_per_ms())
BATCH_SIZE = 128
EMB_SZ = 100
EMB_DIM = 64

class Model(torch.nn.Module):
    """
    A toy model for mimicking the communication overhead of sharded embedding
    modules in DLRM models.

    DLRM models can be trained in a DDP-like fashion, where each trainer
    receives different batches (embedding indices in this example). Since the
    embeddings are sharded across trainers, the lookup process involves (1)
    routing the indices to the trainer that possesses the corresponding
    embeddings (2) performing local lookup (3) routing the embedding lookup
    result back.

    The toy model doesn't actually perform index/result routing. It simply
    uses torch.cuda._sleep() to mimic the cost of the communication op (i.e.
    a2a).
    """

    def __init__(self):
        super().__init__()
        self.local_embedding = torch.nn.Embedding(EMB_SZ, EMB_DIM)

    def forward(self, indices: torch.Tensor):
        local_indices = self._route_indices(indices)
        result = self.local_embedding(local_indices)
        return self._route_result(result)

    def _route_indices(self, batch: torch.Tensor, non_blocking=False):
        """
        This can be parallelized across different batches since it's model
        weight independent.

        Why not run this in dataloader/datamodule?
        - The routing logic depends on how model is sharded
        - Putting this in data preprocessor changes the semantic of the model
        """
        torch.cuda._sleep(CYCLES_PER_MS * 1_000)
        if not non_blocking:
            torch.cuda.synchronize()
        return batch

    def _route_result(self, result: torch.Tensor, non_blocking=False):
        torch.cuda._sleep(CYCLES_PER_MS * 1_000)
        if not non_blocking:
            torch.cuda.synchronize()
        return result

def get_dataloader():
    for _ in range(10):
        yield torch.randint(EMB_DIM, [BATCH_SIZE])

 def non_parallelized():
     dataloader = get_dataloader()
     for batch in dataloader:
         m(batch)

 def parallelized():
     dataloader = get_dataloader()

     comm_stream = torch.cuda.Stream()
     batch_i_ready = torch.cuda.Event()

     # Route the first batch and kick off the routing for the second batch
     with torch.cuda.stream(comm_stream):
         batch_i = next(dataloader)
         m._route_indices(batch_i, non_blocking=True)
         batch_i_ready.record()

     # Invariant: the routing for batch[i] has been kicked off
     done = False
     while not done:
         with torch.cuda.stream(comm_stream):
             batch_ip1_ready = torch.cuda.Event()
             try:
                 batch_ip1 = next(dataloader)
                 m._route_indices(batch_ip1, non_blocking=True)
                 batch_ip1_ready.record()
             except StopIteration:
                 done = True

         batch_i_ready.wait()

         result = m.local_embedding(batch_i)
         m._route_result(result)

         batch_i = batch_ip1
         batch_i_ready = batch_ip1_ready

 if __name__ == "__main__":
     torch.cuda.set_device(torch.device("cuda:0"))

     m = Model()

     begin_ts = time.time()
     non_parallelized()
     print(f"The non-parallelized version took {int(time.time() - begin_ts)} seconds")

     begin_ts = time.time()
     parallelized()
     print(f"The parallelized version took {int(time.time() - begin_ts)} seconds")

Dear @yifuwang,
I have been thinking about the cleanest way to do this with Lighting.
Mind having a look at https://github.com/PyTorchLightning/pytorch-lightning/pull/8700 ?
This line should have to be moved to LightingModule on_after_batch_transfer hook for this work properly.
m._route_indices(batch_ip1, non_blocking=True)

This is just an idea and would like to get your thoughts on this first design.
Best,
T.C

Dear @tchaton,
Thanks for looking into this and coming up with a proposal so quickly!
I really like the idea of having Lightning abstract away this type of parallelism. However, as we discussed earlier, for FB's use case, we still need the flexibility to experiment with different parallelization strategies (even tradeoff between model accuracy and training performance). I'd like to propose that we still add the flavor of training_step with data iterator access (maybe as an experimental API), while pursuing a more user friendly solution (e.g. #8700) for more general use cases. How does it sound?
If you are okay with this, I'd like to give training_step with data iterator access a shot.
Best,
Yifu

Dear @yifuwang,
@kaushikb11 made some extra progress on #8700.
The feature is currently supported only for the GPU accelerator.
We were thinking num_prefetch_batches=-1 could used to request the iterator to be passed directly to the training_step as required by FB.
This assumes fetching, moving data to device (using Lightning utilities) is left to the user control.
One limitation for this approach for Lightning is that the provided batch_idx passed to the training_step can't be inferred and should be provided as None.
Feel free to collaborate with @kaushikb11 on extending Lightning to pass the iterator directly :)
Best,
T.C

I was going back and found some interesting issues previously raised in lightning which discussed this:

https://github.com/PyTorchLightning/pytorch-lightning/issues/2611
https://github.com/PyTorchLightning/pytorch-lightning/issues/1316
https://github.com/PyTorchLightning/pytorch-lightning/pull/1521
https://github.com/PyTorchLightning/pytorch-lightning/issues/4803


Display copy&pasteable commands

The previous commands would fail if a directory contained a space (for example, with the default ".../Tarsnap Backup Inc./..."). QProcess automatically escapes spaces in arguments before it runs
a command, but unbelievably there is no way to access the "escaped" arguments.  (QProcess.arguments() returns the un-escaped strings)
Adding double-quotes around each argument seems like a safer solution than trying to escape spaces.  I admit it that it looks weird to see things like
/usr/bin/tarsnap "--version"

but I think that's an acceptable sacrifice (although clearly not ideal!).
NOTE: the actual command that QProcess runs is not changed by this commit.  This commit only changes what is displayed on the command-line and console log.

I think this is an "acceptable" solution, but I don't like it.  I'm very open to other ideas!

Two thoughts:

Adding "double quotes" around an argument doesn't make it safe to copy and paste into a terminal: The characters $ ` \ " will all result in bad things happening.  I'd suggest using single quotes -- but you'll still need to add additional escaping for single quote characters.

Rather than escaping all strings, how about ignoring any argument which matches /[0-9a-z-A-Z/._-]+/ ?

A couple more thoughts to take into consideration:

QProcess doesn't need to escape the argument list. All it does is convert the QStringList to an argv** and pass that via execvp. The shell is not involved.

Escaping single quotes inside single quotes is not possible, at least in Bash:
http://www.gnu.org/software/bash/manual/bashref.html#Single-Quotes
With double quotes we still have to escape some more special chars as Colin suggested. We also need to take into consideration other shell variants? Or just settle with Bash-alike.

The command line is also shown in Archive details widget. For archives just created the command is the one issued by the application, for archives updated from the remote it will be the one from  tarsnap --list-archives -vv, which suffers from the same non-safe to copy paste issue.

On 10/20/16 01:47, Shinnok wrote:

A couple more thoughts to take into consideration:

QProcess doesn't need to escape the argument list. All it does is convert
the QStringList to an argv** and pass that via execvp. The shell is not
involved.

Right, I don't know the context for this but I assume the idea was to escape
the commands for display somewhere, not for when the GUI is invoking them
directly?

Escaping single quotes inside single quotes is not possible, at least in Bash:
http://www.gnu.org/software/bash/manual/bashref.html#Single-Quotes
With double quotes we still have to escape some more special chars as
Colin suggested. We also need to take into consideration other shell
variants? Or just settle with Bash-alike.

You can't escape single quotes inside single quotes, but you can end a quoted
section, include an escaped quote, and then start a new quoted section.  For
example, 'foo$bar'''baz' gets unescaped by the shell into foo$bar'baz .
--
Colin Percival
Security Officer Emeritus, FreeBSD | The power to serve
Founder, Tarsnap | www.tarsnap.com | Online backups for the truly paranoid

Background: tarsnap-gui displays the commands it's running on the console, and in a "console log" within the application.  I was trying to figure out what these lines did:
QRegExp fileRx("^(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+)\\s+(\\S+\\s+\\S+\\s+\\S+)\\s+(.+)$");
foreach(QString line, _listing.split('\n', QString::SkipEmptyParts))

so I thought I'd copy&paste the command that produced the listing and "step through" it manually.  However, those commands are printed without any escaping, so I got things like (for example):
$ /usr/bin/tarsnap --no-default-config --keyfile /home/td/.local/share/Tarsnap Backup Inc./Tarsnap/gin-2016-09-19-10-06-54.key --cachedir /home/td/.cache/Tarsnap Backup Inc./Tarsnap --print-stats --no-humanize-numbers
tarsnap: Must specify one of -c, -d, -r, -t, -x, --list-archives, --print-stats, --fsck, --fsck-prune, or --nuke

(this is due to the spaces in "Tarsnap Backup Inc.")
PS: I see now that the archive contents are stored offline, so in this case it wouldn't have helped even if the command line was escaped.  But as a general note, I think that if we're going to display command lines, they should be usable.

Are we expecting single quotes inside directory names?  My instinct is that at the moment, it's not worth supporting copy&pasteable commands if somebody stores their Tarsnap key in a directory named Drizzt Do'Urden.  Note that the GUI will still function perfectly well.

I think the only plausible case where a single quote might appear is in the user supplied backup paths and that is mostly expected on a platform like Windows. This is a matter of best-effort on our side, the alternative would be ending an escape sequence like Colin suggested, but then that will make the commands hard to look at. :-)

Updated PR.  This version does not escape the "archive creation command", but all commands shown in Help->Console log and the xterm window are properly escaped (assuming that no arguments contain a single quote).


Compiler: move model_f(;data=Dict()) from compiler to utilities

The current compiler generates a sugar syntax that allows passing all data to a model using a dictionary. This feature is similar with the interface used by the Stan.jl. Implementing this feature at a compiler level is probably not necessary. Perhaps we should move this to utility functions, e.g.

model_f(@data data_dict)

This requires implementing a macro named @data which will act similarly as (x...).

Specifically on the manipulation of kwargs, look at MacroTools.jl. Specifically the stuff towards the bottom on function-expression de-structuring. I've found this package indispensable when doing any metaprogramming.

Thanks, @willtebbutt I'll have a look at it. I'm sure there is a better way then how I do it currently.

We can now do the following:
julia> f(x, y) = 2*x*y^2
julia> dict = Dict(:x => 1, :y => 10)
julia> f( Turing.data(dict, [:x, :y])... )
200
julia> Turing.data(dict, [:x, :y])
(x = 1, y = 10)
julia> f( Turing.data(dict, [:y, :x])... )
20
julia> Turing.data(dict, [:y, :x])
(y = 10, x = 1)
julia> Turing.data(dict, [:x, :y, :z])
ERROR: AssertionError: ...

which is a cheap solution to the problem. Note that the ordering of the symbols in the second argument of Turing.data is deciding the ordering of the entries in the tuple, see last lines.

This looks good!


Project boiler plate

Description
This ensures that laravel is used as the backend and React is used on the frontend.
All pull requests should be related to open issues. Indicate the issue(s) bellow and remove this line:
Fixes #2  #3
How Has This Been Tested?
This was tested locally. an attempt to test it on heroku was not successful due to permission issues.
Checklist:

[ x] My code follows the style guidelines of this project
[x ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have added necessary inline code documentation
[ ] I have added tests that prove my fix is effective and that this feature works
[ ] New and existing unit tests pass locally with my changes

 Here is an overview of what got changed by this pull request:

Issues
======
- Added 157

Complexity increasing per file
==============================
- app/Http/Controllers/Auth/ConfirmPasswordController.php  1
- database/migrations/2014_10_12_100000_create_password_resets_table.php  1
- database/seeds/DatabaseSeeder.php  1
- app/Http/Middleware/Authenticate.php  2
- resources/js/components/Example.js  1
- app/Providers/RouteServiceProvider.php  1
- app/Http/Controllers/Auth/LoginController.php  1
- app/Providers/AppServiceProvider.php  1
- database/migrations/2014_10_12_000000_create_users_table.php  1
- app/Providers/EventServiceProvider.php  1
- database/migrations/2019_08_19_000000_create_failed_jobs_table.php  1
- app/Http/Middleware/RedirectIfAuthenticated.php  2
- app/Providers/AuthServiceProvider.php  1
- app/Http/Controllers/Auth/RegisterController.php  1
- app/Providers/BroadcastServiceProvider.php  1
- tests/Feature/ExampleTest.php  1
- app/Http/Controllers/HomeController.php  1
- app/Http/Controllers/Auth/VerificationController.php  1
- app/Exceptions/Handler.php  1
- app/Console/Kernel.php  1
- tests/Unit/ExampleTest.php  1

See the complete overview on Codacy


Issue #27606: Minor Bug fixes, mostly renaming

Please provide a description for what this PR is for.
Minor bug fixes  fix #27606
And to help us figure out who should review this PR, please
put an X in all the areas that this PR affects.
[ ] Configuration Infrastructure
[ ] Docs
[ ] Installation
[ ] Networking
[ ] Performance and Scalability
[ ] Policies and Telemetry
[X] Security
[ ] Test and Release
[ ] User Experience
[ ] Developer Infrastructure
Pull Request Attributes
Please check any characteristics that apply to this pull request.
[X] Does not have any changes that may affect Istio users.

Hi @shankgan. Thanks for your PR.
I'm waiting for a istio member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.
Once the patch is verified, the new status will be reflected by the ok-to-test label.
I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

cherrypick/release-1.8

/ok-to-test

@shankgan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

release-notes_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test release-notes_istio

integ-multicluster-k8s-tests_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/cherrypick/release-1.8

/test integ-multicluster-k8s-tests_istio

/test release-notes_istio

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test integ-multicluster-k8s-tests_istio

release-notes_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test release-notes_istio

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-multicluster-k8s-tests_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test integ-multicluster-k8s-tests_istio

release-notes_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test release-notes_istio

@shankgan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-pilot-k8s-tests_istio

@shankgan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-pilot-k8s-tests_istio

@shankgan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-pilot-k8s-tests_istio
8eaefea10c595c56a2d63cc1c380f70cad08f4c3
link
/test integ-pilot-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
1ef66d02ffec4f63187a44fd6a2f49cebe216ef3
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-telemetry-k8s-tests_istio

@shankgan: The following test failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
1ef66d02ffec4f63187a44fd6a2f49cebe216ef3
link
/test integ-telemetry-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@shankgan: The following tests failed, say /retest to rerun all failed tests:

Test name
Commit
Details
Rerun command

integ-telemetry-k8s-tests_istio
1ef66d02ffec4f63187a44fd6a2f49cebe216ef3
link
/test integ-telemetry-k8s-tests_istio

integ-multicluster-k8s-tests_istio
907853c36ba81dd30dbb0cc2de94f8032578cd09
link
/test integ-multicluster-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test integ-telemetry-k8s-tests_istio

/test integ-multicluster-k8s-tests_istio

In response to a cherrypick label: new pull request created: #28280


generalize n0n2_LCT propagator to accept an array of n0 and an array of n2 values

This requires editing the element.py crystal class and then the n0n2_lct propagator and adding some logic to determine whether a single n0 n2 value should be used or the array of values.


[Bug]: table title hard to see when Exporting PDF

Bug Description
table title hard to see when exporting PDF
Relevant Screenshot
No response
To Reproduce
No response
Obsidian Version
0.13.19
Installer Version
0.13.19
Which editor are you using?

[ ] New Editor
[ ] Legacy Editor

In which mode(s) does the problem occur? (if applicable)

[ ] Source (Edit)
[ ] Reading (Preview)
[ ] Live Preview

Which operating system?

[ ] macOS
[ ] Linux
[ ] Windows
[ ] iOS
[ ] Android

Plugin-Specific Problems

[ ] I took note of the specific instructions regarding those plugins.

Checklist

[X] I updated to the latest version of the theme (Settings → Appearance → Themes - Manage → Update).
[X] I made sure that the issue is related to this theme, by checking that the problem does not occur with other themes or the default theme.
[X] In case I am missing an UI element, I have checked that it cannot be re-enabled with the Style Settings plugin.

Hi! As you may have heard, the upcoming Obsidian update 0.16 has a massive amount of changes for theme development, necessitating a complete rewrite of Shimmering Focus. This means:

A lot of bugs and feature requests will be obsolete. with Obsidian 0.16 and the rewritten Shimmering Focus.
As the theme has over 15,000 lines of CSS, the rewriting will take a considerable amount of time. Recreating the core features alone will likely take me a while, so I simply will not have much time to implement many feature requests, since the recreation of the past core features has priority.

Basically, this is the reason why I am – for now – closing this issue. If your bug still exists / your feature request is still relevant in the upcoming version 3.0 of Shimmering Focus, please comment here and I will re-open it. 🙂


“Account Number” alignment is not proper when the user taps on  Hamburger Menu from the Home screen.

Frequency: 100%
Repro on build version: Play store build V1.10.1
Does not repro version App store build V1.10.1
Repro on devices: Samsung Galaxy A21s(10.0),Samsung Galaxy M31(10.0)
Does not repro devices Vivo V11 pro (10.0)
Repro Steps:

Launch the application & login without phone  number.
Go to the home screen & tap on “Hamburger Menu” & scroll down.
Observed that,
Current Behavior:  Account number alignment not proper.
Expected Behavior:  Account number alignment should be proper.
Impact: Bad user experience.
Attachment: AccountNumberAllignmentNotProper.png

Hi  I have verified this issue using latest  Android Internal Build V1.11.1(1004294339) & observed that issue is still reproduced.
Verified on Devices: Samsung Galaxy A21s(10.0)
Does Not Repro on Devices: Samsung Galaxy M31(10.0)


Allow more colors on a theme

Please describe the feature you would like to request.
It would be great if we could have more than just primary, accent and warn colors.
What is the use-case or motivation for this proposal?
It is very hard to use only 3 colors on the entire application, and even tho it is possible to use css classes, when you create an application supporting multiple themes, it gets hard to maintain it as application grows.
Is there anything else we should know?
It also would be great if we could have an extra directive to define what color shade from the palette we want to use as base.
For Example.: <button color="accent" colorShade="300">Button</button>

💯 agreed. The workarounds to hack in additional colors adds a lot of bloat

We unfortunately have no plans to expand the theming system until we're able able to switch to CSS variables (when we can drop support for IE11).


Fix #287 - Eating of whitespace at start of line

In addLine() we return the substring "Foo()" when it should be "  Foo()".
This patch removes taking the substring and instead returns the line as is.
The prior if branch (here) seem to suggest that this is the expected behavior since it converts tabs to spaces, yet does not remove those.

Doesn't this change fail a lot of tests?

Doesn't this change fail a lot of tests?

Yes it does. I'm not sure what problem it's trying to solve, I've replied on #287


NETOBSERV-1672: add cluster & zone to predefined metrics

Description
Add cluster & zone to predefined metrics
They're only added when those features are enabled.
Dependencies

n/a
Checklist
If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

[x] Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
[ ] Does this PR require product documentation?

[ ] If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.

[ ] Does this PR require a product release notes entry?

[ ] If so, fill in "Release Note Text" in the JIRA.

[ ] Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.

[ ] If so, make sure it is described in the JIRA ticket.

QE requirements (check 1 from the list):

[x] Standard QE validation, with pre-merge tests unless stated otherwise.
[ ] Regression tests only (e.g. refactoring with no user-facing change).
[ ] No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

@memodi did you use the latest generated image, not the first one? ie. a3ff45b and not c20ce8c. Because I had this same problem at first and I (should have) fixed it in the second build

@memodi did you use the latest generated image, not the first one? ie. a3ff45b and not c20ce8c. Because I had this same problem at first and I (should have) fixed it in the second build

ah, yes, I had used a3ff45b, I'll try with with c20ce8c

thanks @jotak , a3ff45b works.
/label qe-approved


path_open / OFLAGS_CREAT

it seems path_open does not allocate fd when passed OFLAGS_CREAT, does not handle OFLAGS_EXCL
and maybe does not return all possible values in all posible cases ( ERRNO_EXIST / ERRNO_NOTDIR )

Notes from discord:

the fd is not allocated when creating
it should test for file, then if file is not there, test for valid parent and rights to create in it
leads to fopen with mode w/w+ failing right now
needs logic here


Unexpected missing key on 3rd attempt

I've found some surprising (at least to me) behaviour where I can successfully read a key twice, but it fails to find it the third time. My key is variable length, but my value is fixed length. Repro below, this test fails for me against current master.
[TestFixture]
internal class ReproSpanByteKey
{
    class Functions : FunctionsBase<SpanByte, long, long, long, Empty>
    {
        public override void SingleReader(ref SpanByte key, ref long input, ref long value, ref long dst) => dst = value;

        public override void ConcurrentReader(ref SpanByte key, ref long input, ref long value, ref long dst) => dst = value;
        public override void ReadCompletionCallback(ref SpanByte key, ref long input, ref long output, Empty ctx, Status status) { }
    }

    [Test]
    public void Repro()
    {
        var log = Devices.CreateLogDevice(TestContext.CurrentContext.TestDirectory + "/BasicFasterTests.log", deleteOnClose: true);
        var fht = new FasterKV<SpanByte, long>(
            size: 1L << 27,
            new LogSettings
            {
                LogDevice = log,
                MemorySizeBits = 15,
                PageSizeBits = 12,
            });
        var session = fht.For(new Functions()).NewSession<Functions>();

        // write enough records to be flushed to disk
        for (int i = 0; i < 5000; i++)
        {
            var keyString = $"{i}";
            var key = MemoryMarshal.Cast<char, byte>(keyString.AsSpan());
            session.Upsert(SpanByte.FromFixedSpan(key), i);
        }
        session.CompletePending(wait: true);

        // read "5" twice successfully
        Assert.AreEqual(5, ReadKey5());
        Assert.AreEqual(5, ReadKey5());

        // BUT not found on the third time
        Assert.AreEqual(5, ReadKey5());

        long ReadKey5()
        {
            var keyString = $"5";
            var key = MemoryMarshal.Cast<char, byte>(keyString.AsSpan());

            var status = session.Read(key: SpanByte.FromFixedSpan(key), out var unused);

            // key low enough to need to be fetched from disk
            Assert.AreEqual(Status.PENDING, status);

            session.CompletePendingWithOutputs(out var completedOutputs, wait: true);

            var count = 0;
            var value = 0L;
            while (completedOutputs.Next())
            {
                count++;
                ref var completedKey = ref completedOutputs.Current.Key;
                var completedStringKey = new string(MemoryMarshal.Cast<byte, char>(completedKey.AsReadOnlySpan()));
                Assert.AreEqual(Status.OK, completedOutputs.Current.Status);
                Assert.AreEqual(5, completedOutputs.Current.Output);
                value = completedOutputs.Current.Output;
            }
            completedOutputs.Dispose();

            Assert.AreEqual(1, count);
            return value;
        }
    }
}

This was a dispose bug with the new API (CompletePendingWithOutputs) we added last week. Thanks for the catch. Using the older stable API of CompletePending will work just fine as workaround (getting result via the Read completion callback in IFunctions).
The linked PR fixes the issue as well, adding this as a testcase.
Also, note that for the SpanByte based calls to FASTER, as warned in the tooltip for FromFixedSpan, you need to fix the Span for the duration of the synchronous operation, since it points to heap memory (strings) that may shift around due to GC:
                fixed (byte* f = key)
                    session.Upsert(SpanByte.FromFixedSpan(key), i);

                fixed (byte* f = key)
                    var status = session.Read(key: SpanByte.FromFixedSpan(key), out var unused);

Brilliant, thanks!
Good point re fixing, my original key was stackalloc'ed but I seem to have changed that without adding the fixing 👍


Fails to build - 'height' undeclared

I'm using the jwm-git package from the ArchLinux AUR.
Since: https://github.com/joewing/jwm/commit/3c393d0d989f283b58eb610ff6764a0835bf568b
image.c: In function ‘LoadSVGImage’:
image.c:478:10: error: ‘height’ undeclared (first use in this function); did you mean ‘rheight’?
          height = dim.height * xscale;
          ^~~~~~
          rheight

I can confirm this.
Attempting to compile 1670, in my fork of OpenEmbedded, that compiles my entire "Pyro" series of Quirky Linux and Easy OS. Ref: https://github.com/bkauler/oe-qky-src
\"/usr/share/locale\"  misc.c
| ERROR: oe_runmake failed
| image.c: In function 'LoadSVGImage':
| image.c:478:10: error: 'height' undeclared (first use in this function)
|           height = dim.height * xscale;
|           ^~~~~~

librsvg is version 2.40.16
Earlier versions of jwm that compiled ok in same build environment are rev976 and release version 2.3.7.
Regards,
Barry

Thanks! I was building without rsvg support.  This should be fixed now (snapshot 1675).

@ml-
Can this be closed? I have not noticed this problem on my end in building it


Support multiple registries

Similar to #59, an image is limited to a single registry. This is because skopeo uses the image name to determine where to publish the image when copyToRegistry is called (i.e.,my-registry.com/image-name). Since the name can only be set once in the call to buildImage, currently pushing to multiple registries would require using multiple derivations, which in turn would (I believe) change the image hash and not be a suitable method.
Instead, we could add a registries option to buildImage, which automatically prepends the image name with each registry and pushes the image accordingly. I don't proclaim to be an expert on this subject, so correct me if I'm wrong here.

Use case:

Cluster Registry for Ops
Github or Docker Registry for Distribution / Releases

I don't think we should introduce all these attributes (registries, extraTags) related to the image distribution in the image build function.
I introduced the image.copyTo* attributes to cover a simple usecase and it doesn't seem suitable for more complex usecases.
Unfortunately, I didn't find a good idea yet to cover your usecase. Maybe we could introduce a new function (something such as mkDistributionScript) taking an image as argument plus some other optional arguments (such as registries, extraTags). wdyt?

We also can implement these in std's Container Block Type Action since that is what blocktype actions deal with. However, it would be good to store image name(s) and tag(s) in a lazyDerivation.

@nlewo If I make a mkDistributionScript type function, you'd be happy to merge that?

@jmgilman yes, i think adding a new function would definitively be possible.
However, i'm not sure that would solve all your issues, since you would also like to add more attributes to the buildImage function: I'm more skeptical on this part.

As previously discussed, i would prefer to avoid adding tags, names, and registries attributes to the buildImage function, since it's purpose it to build an image and not to push it.
I also understand you want to attach such kind information to the image. So, the meta attribute looks like a good tradeoff :+1:
I'm closing this issue since it has been addressed by #65


test: use T.TempDir to create temporary test directory

A testing cleanup.
This pull request replaces os.MkdirTemp with t.TempDir. We can use the T.TempDir function from the testing package to create temporary directory. The directory created by T.TempDir is automatically removed when the test and all its subtests complete.
This saves us at least 2 lines (error check, and cleanup) on every instance, or in some cases adds cleanup that we forgot.
Reference: https://pkg.go.dev/testing#T.TempDir
func TestFoo(t *testing.T) {
	// before
	tmpDir, err := os.MkdirTemp("", "")
	assert.NoError(t, err)
	defer os.RemoveAll(tmpDir)

	// now
	tmpDir := t.TempDir()
}

@allcontributors please add @Juneezee for test


Dead link

Just to inform that the link http://jellekralt.github.io/Responsive-Tabs/ is returning a 404 (page not found).

Seems to be a GitHub pages issue: https://twitter.com/githubstatus/status/801079562893299712

Should be solved now https://twitter.com/githubstatus/status/801121500044595200


Add tests for service reordering due to annotations

Description:
$title.
Could base these on the WebSocket scenarios.
Related: https://github.com/ballerina-platform/ballerina-lang/pull/15884, https://github.com/ballerina-platform/ballerina-lang/pull/16549

This is deferred.


update crypto (2.3)

Signed-off-by: Wei Liu<EMAIL_ADDRESS>

refer to https://github.com/stolostron/backlog/issues/18857

/lgtm


Add a simple example

This lib looks promising but I am having a hard time picturing the client side flow. Could you add a simple example to make it clear what the user needs to handle?

Yes, some example code would be helpful.


Update PMD4 validations to reflect current muttnik expectations (#1249)

[x] Change DatasetExactlyOneContents -> DatasetMaxOneContents
[x] Change DatasetExactlyOneGraph -> DatasetMinOneGraph
[x] Remove DatasetExactlyOneLabel
[x] Add DatasetExactlyOneTitle
[x] Remove DatasetMaxOneContributor
[x] Remove RecordExactlyOneLabel
[x] Individual tests for modified/new validations
[x] Run PMD4 suite on removed validations

Run tests with clojure -M:test

@Robsteranium I've been through each of these and they seem good.  And I've updated @kieranlives corresponding muttnik PR so it now passes wrt IC-19.
https://github.com/Swirrl/muttnik/pull/1327
This means once that PR is merged, muttnik fixtures will be aligned with these changes.
Are there any other comments on these changes?

I've merged this in to #24.
Thanks!


chore: add noErrorTruncation to the base tsconfig.json

This results in the ts preview of vscode not truncating the hovered information of a type or error. Helpful for inspecting long types.
Follow up to: https://github.com/dai-shi/waku/pull/854#issuecomment-2336516245

Let's merge. Thanks!


Readonly fields and getter-only class members are not compatible when merging declarations

Bug Report

🔎 Search Terms
readonly, getter, get, interface, class, module, interface merging

🕗 Version & Regression Information

This is the behavior in every version I tried, and I reviewed the FAQ for entries about interface merging

⏯ Playground Link

Playground link with relevant code
💻 Code

interface Test4 {
  readonly x: number;
}

class Test4 { // same name, merged interface
  get x() { return 33 }; // ERR duplicate identifier 'x'
}

🙁 Actual behavior
Error on both the getter and the readonly field, Duplicate identifier 'x'.
🙂 Expected behavior

I expected the two declarations to successfully merge, thinking that readonly x: number and get x(): number with no setter function would be compatible as essentially the same type. The Sandbox link has other tests to attempt to demonstrate that TS considers the types compatible in other cases.

thinking that readonly x: number and get x(): number with no setter function would be compatible as essentially the same type.

They're not the same. The one represents a property, the other a better. Note that readonly properties are only readonly via that interface. Interfaces with readonly properties are implicitly compatible with mutable properties. Your getter wouldn't work with that.

I don't have much to add.

@MartinJohns How is your answer to be reconciled with https://github.com/microsoft/TypeScript/issues/3745#issuecomment-118924225, specifically @RyanCavanaugh's statement, "Whether a property is implemented as a field or a getter/setter pair is an implementation detail, not part of the type."
Additionally:

Interfaces, with or without declaring a property readonly, seem to have no way to reference getters in certain crucial contexts, as the following interface merge fails with the same error:
interface Foo {
    x: number;
}
class Foo {
    get x(): number {
        return 0;
    }
}

The comment from 2015 is no longer accurate due to TC39 deciding on [[define]] semantics for class fields

Oh, okay. How recent is TC39? https://github.com/microsoft/TypeScript/issues/40415#issuecomment-689850483


Summary statistics for all regions?

Regine: It would be cool to have this for all 19 regions, esp the attributes but hard to make compact

Also useful: a hypsometry plot for each region.

The idea would be to add a new "additional info" box to each region with the summary plots and statistics.


feat: migrate security hub

added proposal from #70 as well :)
in favor of #350

UPDATE_FROM_LATEST_RELEASE: false was successful: https://github.com/superwerker/superwerker/actions/runs/4226998696/jobs/7348155956


Call Hierarchy view overview ruler overlaps SCM and caller decorators

VSCode Version: 1.42 Insiders
OS Version: Windows 10

Found while testing #89386

Make a change such as add some new callers.

The green SCM addition region overlaps with the caller decoration.

Closing this without any change. Looking at our related usages like word highlights or find matches shows that all "discovery" highlights are rendered in the center-lane. With all of them there is potential for overlap and the other alternative is to not have any ruler highlights at all


Limit maxDepth of list when pasting.

Since RichUtils has provided onTab(event, editorState, maxDepth) to change the depth of a list item, with the ability of limiting max depth. I think there should also be a method to limit  the max depth when pasting, for consistency and for preventing unexpected depth.
Currently I'm fixing it by rewriting the depth with the these code, in Editor.props.onChange.

  onChange = (editorState) => {
    let contentState = editorState.getCurrentContent()

    if (contentState !== this.state.editorState.getCurrentContent()) {
      if (editorState.getLastChangeType() === 'insert-fragment') {
        let blockMap = contentState.getBlockMap()
        const blocks = blockMap
          .filter((block) => block.getDepth() > MAX_LIST_DEPTH)
          .map((block) => block.set('depth', MAX_LIST_DEPTH))

        blockMap = blockMap.merge(blocks)
        contentState = contentState.merge({ blockMap })
        editorState = EditorState.set(editorState, {
          currentContent: contentState
        })
      }
    }

    this.setEditorState(editorState)
  }

I think it could be costly by searching through the whole blockMap.
Thanks!

Edit: It might be a problem of paste, which is so powerful and its generated fragment is not changeable. If I want to limit the using of  some default styles (e.g. list, bold, header-*) , I should also limit them when pasting. blockRenderMap and  handlePastedText will help, but they're not enough.

There could potentially be a configuration prop for paste handling behavior, with this field included. Right now MAX_DEPTH is hardcoded in paste processing code, which is certainly not ideal. I assume you're aiming to have a smaller depth value?

Thanks for reply!

I assume you're aiming to have a smaller depth value?

Yes, sorry for not making it clear.
I just found there are already discusses on it, either a configuration prop (#166) or post process ability for  converFromHTMLToContentBlocks (#380) will help.
I'm closing it, thanks.


ConceptID and reset button feature

Pull request consists

Tested button feature in TestFeatures.js
ConceptID issue solved and tested  in Util.js

@manlikleo there's is something wrong, I see 2085 files changed, can't be true, probably git/github is being confused somehow.  See if you can fix it, else, I can try cloning your fork (is it up-to-date) and merge it with the main repo from my laptop. Please, let me know.

@manlikleo there's is something wrong, I see 2085 files changed, can't be true, probably git/github is being confused somehow. See if you can fix it, else, I can try cloning your fork (is it up-to-date) and merge it with the main repo from my laptop. Please, let me know.
EDIT: I found it, you've added a copy of the whole codebase into .github. Delete it.
Moreover,

Your fork is behind the main repo, we have added a few changes in the last days, please merge from
upstream and check again that everything works.

This merge before committing changes (to a fork or a main repo) should be routinely done, it avoids
introducing issues due to lack of synchronisation (worse, without it, you risk committing old code).

Add .vscode/ to .gitignore and remove this dir from the repo. We don't upload IDE configurations, since
these contain settings that are developer-specific, everyone has their own copy of these files and they
don't need to be shared (sometimes it's even usafe if they contain things like passwords).

testFeature.js, what is this exactly?

a temp file, used during tests => in this case, delete it (and possibly move the code somewhere else)
or something that should become permanently part of production code => in this case, give it better
names and possibly add comments.

@manlikleo there's is something wrong, I see 2085 files changed, can't be true, probably git/github is being confused somehow. See if you can fix it, else, I can try cloning your fork (is it up-to-date) and merge it with the main repo from my laptop. Please, let me know.
EDIT: I found it, you've added a copy of the whole codebase into .github. Delete it.
Moreover,

Your fork is behind the main repo, we have added a few changes in the last days, please merge from
upstream and check again that everything works.

This merge before committing changes (to a fork or a main repo) should be routinely done, it avoids
introducing issues due to lack of synchronisation (worse, without it, you risk committing old code).

Add .vscode/ to .gitignore and remove this dir from the repo. We don't upload IDE configurations, since
these contain settings that are developer-specific, everyone has their own copy of these files and they
don't need to be shared (sometimes it's even usafe if they contain things like passwords).

testFeature.js, what is this exactly?

a temp file, used during tests => in this case, delete it (and possibly move the code somewhere else)
or something that should become permanently part of production code => in this case, give it better
names and possibly add comments.


Finalize logging analyzers

Consider adding analyzers for DefineMessage
Reference analyzers from our projects
Decide how are they going to be shipped
/cc @glennc

Where are these analyzers published?

Right now they are published to https://dotnet.myget.org/F/aspnetcore-ci-dev/api/v3/index.json feed. https://dotnet.myget.org/feed/aspnetcore-ci-dev/package/nuget/Microsoft.Extensions.Logging.Analyzers

Will they work on VS2015?

@pakrym, still no plans to release this?

No, not in 2.1

😢

@pakrym - are we shipping them in "experimental" form so that people can easily try them out? I thought I heard someone say that the other day.

Yes, we would continue to ship them in the same form they are now, but wouldn't do the same thing we did with MVC and EF analyzers (reference from packages themselves)

I think these analyzers should be a little more extensive than they are now. It would also be great if they supported structured logging templates. The structured message analyzer rule can be 'none' severity by default and only enabled if users want them.


[DCOS-53557][COPS-4781] Expose Kibana web server through an nginx proxy.

This was done to address the issue surfaced by COPS-4781: accessing the Kibana service URL resulted in a redirect to an invalid URL. Namely:
GET /service/kibana/ => /service/kibana/service/kibana/app/kibana

The expected redirect should have been:
GET /service/kibana/ => /service/kibana/app/kibana

This happened because of a breaking change in Kibana 6.x which caused requests to the Kibana web server hitting the root path (/) resulted in an invalid redirect, depicted above.
This could be the place where this bug was introduced:
https://github.com/elastic/kibana/pull/21408/files#diff-230bb53cded88451213fc31d8a4b0774L103
The solution here was to add an nginx proxy in between Admin Router and the Kibana web server, that rewrites requests that would hit the Kibana web server on the root path to hit it on /app/kibana instead.
I also took the opportunity to extract the Kibana Marathon app command to an actual bash script, which is much more maintainable than the previous one-line cmd directive.

@samvantran thanks for taking a look. This change was partially covered by the existing tests, but I added two additional assertions: 17acf625f3ac91c86b6d97e5ed25bc381ba7f00a
Regarding the possibility of not needing the nginx proxy because of a possible fix for Kibana on future versions, it might still make sense to keep it because it makes it possible for us to fix the issue detailed in this ticket: https://jira.mesosphere.com/browse/DCOS-46801

Thanks for additional assertions. @rishabh96b please take a look too !

The Elastic strict build passed, but the permissive build failed failed on a known flake: https://teamcity.mesosphere.io/viewLog.html?buildId=1878143&buildTypeId=DataServices_Elastic_IntegrationTests_DCOS_master_Permissive_PR&guest=1
Merging this for now.


Avoid posting comment multiple times in same PR

The fact that the PR comment gets posted on every push if there's 1 (older) commit in the PR that's unsigned can result in noisy PRs.
So instead of posting the PR comment over and over, we'll now check the PR comments and only post the comment if it hasn't been posted earlier. This won't impact the exit code that gets returned.
Resolves #8
Logs of test runs can be found here: unsigned commit that results in comment and second unsigned commit that skips the comment.

Can I check on the status of this PR? Friendly bump!

Thank you for improving the experience. This looks great to me. I do appreciate you providing logs for tests runs showing how this adjustment positively impacts the action.
I have one question (non-blocking). From my understanding, the comment message isn't updated if it already exists. However, how can a user who adds more commits to a PR know that the specific comment is still up-to-date and relevant? Having the pipeline fail is one way to achieve it, but is this enough? 🤔
Approved. ✅  💪🏻

However, how can a user who adds more commits to a PR know that the specific comment is still up-to-date and relevant? Having the pipeline fail is one way to achieve it, but is this enough?

That's a bit of a trade-off, indeed. But the first comment is already so in your face that I think having the check fail will later on will be enough.


HOWTO join grafadruid.slack.com

When I click on https://grafadruid.slack.com, it expects me to have an account and I do not have. I would like to know more about the project as we use golang heavily and is evaluating druid as our OLAP system.

Hello @vigith !
Yes, you need an account to use Slack. You can easilly create one in few clicks.
Otherwhise, I just enabled "Discussions" feature here on Github, feel free to use it: https://github.com/grafadruid/go-druid/discussions !
We'll be happy to help :)

Hello @vigith !
Yes, you need an account to use Slack. You can easilly create one in few clicks.
Otherwhise, I just enabled "Discussions" feature here on Github, feel free to use it: https://github.com/grafadruid/go-druid/discussions !
We'll be happy to help :)

May be I need an invitation to join the grafadruid.slack.com? Thank you for enabling discussion.

May be I need an invitation to join the grafadruid.slack.com? Thank you for enabling discussion.

Sorry! Here it is: https://join.slack.com/t/grafadruid/shared_invite/zt-kxaiq5gm-c6akAiceMzgHvUBV~KXeiA
Replaced the link on the project description. That stuff will need some automation I bet since the link is active for 30 days / 2K users. Or maybe another place than Slack. Something to dig later :)

Sorry! Here it is: https://join.slack.com/t/grafadruid/shared_invite/zt-kxaiq5gm-c6akAiceMzgHvUBV~KXeiA
Replaced the link on the project description. That stuff will need some automation I bet since the link is active for 30 days / 2K users. Or maybe another place than Slack. Something to dig later :)


Add database connect timeout to warehouse settings

We would need to add this as a standard setting across all the warehouses.
We would allow using of this property and override it to something short when we do datasource validation to avoid defaulting to long periods of time, but customers should also be able to utilize it if they wish to do so.
Currently for PostgreSQL we do already this, but it is clumsily implemented via *args/**kwargs so that needs to be removed

Guidelines on implementation:
Scanner now uses *args, **kwargs to pass the connection_timeout_sec in soda_scanner_web_app.py
warehouse = Warehouse(warehouse_yml_parser.warehouse_yml, connection_timeout_sec=5)

This should be appended into the connection details above
            warehouse_yml_dict = self._create_warehouse_yml_dict(
                {KEY_CONNECTION: datasource_json}
            )

So that *args, **kwargs can be removed from the Warehouse constructor in Soda SQL
class Warehouse:

    def __init__(self, warehouse_yml: WarehouseYml, *args, **kwargs):


The description provided for Orientation property of "StackPanel2" in the example is wrong.

The "StackPanel2" defined in the code has Orientation property assigned to "Vertical". But the TextBlock used to describe the StatckPanel orientation mentions "StackPanel.Orientation="Horizontal" which is wrong.

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: a01663b5-c69f-20d0-5d02-e216bd54565f
Version Independent ID: a13e9d64-18ca-dc0e-0afe-211d921a98f6
Content: Alignment, Margins, and Padding Overview - WPF .NET Framework
Content Source: dotnet-desktop-guide/framework/wpf/advanced/alignment-margins-and-padding-overview.md
Product: dotnet-framework
Technology: dotnet-wpf
GitHub Login: @adegeo
Microsoft Alias: adegeo

Thank you for opening this issue. I'll add it to the backlog. If you would like to contribute, you can submit a fix by editing the original article. Click on the Content Source link at the bottom of your original comment. To learn how to edit, see the Editing files in a repository article from GitHub.
Thanks again!

This is my first ever "Help Wanted" issue and my first contribution to open source in general!  I submitted a pull request.


Flutter crash report.

Flutter crash report.
Please report a bug at https://github.com/flutter/flutter/issues.
command
flutter run
exception
FileSystemException: FileSystemException: Failed to decode data using encoding 'utf-8', path = 'G:\person\flutter\flutter_vscode_demo\android\local.properties'
#0      _File._tryDecode (dart:io/file_impl.dart:564:7)
#1      _File.readAsStringSync (dart:io/file_impl.dart:584:7)
#2      ForwardingFile.readAsStringSync (package:file/src/forwarding/forwarding_file.dart:96:16)
#3      ErrorHandlingFile.readAsStringSync.<anonymous closure> (package:flutter_tools/src/base/error_handling_io.dart:221:22)
#4      _runSync (package:flutter_tools/src/base/error_handling_io.dart:573:14)
#5      ErrorHandlingFile.readAsStringSync (package:flutter_tools/src/base/error_handling_io.dart:220:12)
#6      new SettingsFile.parseFromFile (package:flutter_tools/src/base/utils.dart:135:36)
#7      updateLocalProperties (package:flutter_tools/src/android/gradle_utils.dart:202:29)
#8      AndroidProject.ensureReadyForPlatformSpecificTooling (package:flutter_tools/src/project.dart:938:5)
#9      FlutterProject.ensureReadyForPlatformSpecificTooling (package:flutter_tools/src/project.dart:302:21)
<asynchronous suspension>
#10     FlutterCommand.verifyThenRunCommand (package:flutter_tools/src/runner/flutter_command.dart:1175:7)
<asynchronous suspension>
#11     FlutterCommand.run.<anonymous closure> (package:flutter_tools/src/runner/flutter_command.dart:1043:27)
<asynchronous suspension>
#12     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#13     CommandRunner.runCommand (package:args/command_runner.dart:196:13)
<asynchronous suspension>
#14     FlutterCommandRunner.runCommand.<anonymous closure> (package:flutter_tools/src/runner/flutter_command_runner.dart:284:9)
<asynchronous suspension>
#15     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#16     FlutterCommandRunner.runCommand (package:flutter_tools/src/runner/flutter_command_runner.dart:232:5)
<asynchronous suspension>
#17     run.<anonymous closure>.<anonymous closure> (package:flutter_tools/runner.dart:62:9)
<asynchronous suspension>
#18     AppContext.run.<anonymous closure> (package:flutter_tools/src/base/context.dart:150:19)
<asynchronous suspension>
#19     main (package:flutter_tools/executable.dart:91:3)
<asynchronous suspension>

flutter doctor
[✓] Flutter (Channel stable, 2.2.2, on Microsoft Windows [Version 10.0.18363.1500], locale zh-CN)
    • Flutter version 2.2.2 at D:\programe_install\flutter_windows_2.2.2-stable\flutter
    • Framework revision d79295af24 (2 weeks ago), 2021-06-11 08:56:01 -0700
    • Engine revision 91c9fc8fe0
    • Dart version 2.13.3

[!] Android toolchain - develop for Android devices (Android SDK version 30.0.3)
    • Android SDK at D:\Android_SDK
    • Platform android-30, build-tools 30.0.3
    • Java binary at: D:\programe_install\AndroidStudio\jre\bin\java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[✓] Chrome - develop for the web
    • Chrome at C:\Program Files (x86)\Google\Chrome\Application\chrome.exe

[✓] Android Studio (version 4.0)
    • Android Studio at D:\programe_install\AndroidStudio
    • Flutter plugin version 51.0.1
    • Dart plugin version 193.7547
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b01)

[✓] Connected device (3 available)
    • Android SDK built for x86 (mobile) • emulator-5554 • android-x86    • Android 7.1.1 (API 25) (emulator)
    • Chrome (web)                       • chrome        • web-javascript • Google Chrome 86.0.4240.111
    • Edge (web)                         • edge          • web-javascript • Microsoft Edge 91.0.864.59

! Doctor found issues in 1 category.

Closing this as duplicate of https://github.com/flutter/flutter/issues/50723


Test coverage fails to run on CI

Successfully installed python-coveralls PyYAML coverage sh
Cleaning up...
$ coverage run --include=ensime_launcher/__init__.py,rplugin/python/ensime.py spec/ensime.py
EE.....
======================================================================
ERROR: test_ensime (__main__.TestEnsime)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "spec/ensime.py", line 257, in test_ensime
    assert(self.ensime.with_current_client(lambda c: None) == None)
  File "spec/../rplugin/python/ensime.py", line 525, in with_current_client
    c = self.current_client(create, quiet, allow_classpath_file_creation)
TypeError: fake_current_client() takes no arguments (3 given)

======================================================================
ERROR: test_ensime_client (__main__.TestEnsime)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "spec/ensime.py", line 167, in test_ensime_client
    for typehint in ["NewScalaNotesEvent", "SymbolInfo", "IndexerReadyEvent", "AnalyzerReadyEvent", "BasicTypeInfo", "StringResponse", "CompletionInfoList"]]
  File "spec/../rplugin/python/ensime.py", line 350, in handle_payload
    self.show_type(payload)
  File "spec/../rplugin/python/ensime.py", line 366, in show_type
    if payload["typeArgs"] != []:
KeyError: 'typeArgs'

----------------------------------------------------------------------
Ran 7 tests in 0.227s

I'll look into this later today unless someone jumps on it.

Hi @ChrisCoffey, don't care about the tests, they've been failing for ages and we have to re-engineer the test suite. See #136.


Upload 2.2.0 to maven central

For maven users, the latest version is still 2.1.0:
https://mvnrepository.com/artifact/net.jafama/jafama

Hello.
The new version, 2.3, is now on central:
https://mvnrepository.com/artifact/net.jafama/jafama/2.3.0
Sorry for being so lazy for so long, but as I said for another pull request, I'm mostly focused on some other big project ATM, and I was missing will for context switching, especially because I had to sort out my messy notes about what I did to upload to central the first time.
There was also nothing very useful in 2.2, just a few cornercase methods and lazy tables initialization (which can have its drawbacks, which is why I added initTables() method along with it).


String literal types failing in waterline

See DefinitelyTyped/types/waterline; I added casts to get these to work, but ideally we could just write "email" instead of "email" as "email".

The waterline types contain
    export type BaseAttribute<T> = AttributeValidations & {
        type?: string;
        primaryKey?: boolean;
        unique?: boolean;
        required?: boolean;
        enum?: Array<T>;
        size?: number;
        columnName?: string;
        index?: boolean;
        defaultsTo?: T | DefaultsToFn<T>;
    }
    export type StringAttribute = BaseAttribute<string> & {
        type: "string";
    }

string | "string" is super likely to get simplified to string the type gets mapped over in any way (as only contextual type traversal is going to preserve that union right now). But all is not lost, the waterline types can be rewritten:
    export type BaseAttribute<T, TType extends string = string> = AttributeValidations & {
        type?: TType;
        primaryKey?: boolean;
        unique?: boolean;
        required?: boolean;
        enum?: Array<T>;
        size?: number;
        columnName?: string;
        index?: boolean;
        defaultsTo?: T | DefaultsToFn<T>;
    }
    export type StringAttribute = BaseAttribute<string, "string">;

IMO, the types are actually better defined this way (certainly more concise) so the types can be made to continue working, but looking into if we must preserve our behavior for these packages is still an issue.

That's an intersection, so shouldn't it be string & "string" not string | "string"?

You're probably right that it's better without the intersection types though.

That's an intersection, so shouldn't it be string & "string" not string | "string"?

I think because the field is optional in the original type you end up with an intersection applied over a union that then has redundant literals removed.

Yep, definitely a bug caused by the intersection type. I know the issue and will have a fix up momentarily.


Prepare v1.13 stable branch

Notes:

v2alpha1 schema version is bumped down to match the v2 schema version. The minor version was inadvertently bumped in the tree recently, but we only need to bump the minor when we
create a new minor branch. Some scripts also expect that the schema versions match between v2 and v2alpha1.
Branching the build image workflows and CI jobs is complicated and not documented at all currently. Most other steps are documented in https://github.com/cilium/cilium/pull/22614.

Need to check:

[ ] Is it OK to delete all these conformance tests from this branch? Will we lose coverage in kind, ingress, etc.
[ ] How do we update the .github/workflows/build-images-* for a branch?

v1.12 seemed to delete a bunch of caching stuff, was that intentional?
Big difference: v1.13 now has cosign stuff

[ ] Some workflows seem to want to check out master branch first, should they? Do we delete those bits?
[ ] Can we delete .github/workflows/build-images-releases.yaml since the master branch workflow matches all release versions?
[ ] Update the branch protection rules for the latest v1.13 GHA and Jenkins workflows
[ ] Continue from Step 12 of the Branching instructions

So far this was prepared by following https://docs.cilium.io/en/latest/contributing/release/feature/#on-freeze-date . However, comparing with v1.12 commit 9b9caf949e3c88469b10bc0bd1c89603b00f762c I can see that the previous stable branching removed many more files than are documented, so we may need to do more .github cleanup.
This commit now:
$ git show --stat HEAD
commit 0d114f2f3e6e4faf96c26826706703fd47c187e0 (HEAD -> pr/prepare-v1.13, upstream/pr/prepare-v1.13)
Author: Joe Stringer<EMAIL_ADDRESS>Date:   Wed Dec 7 23:12:56 2022 -0800

    Prepare v1.13 stable branch

    The only curious thing here is to bump the v2alpha1 schema version down
    to match the v2 schema version. The minor version was inadvertently
    bumped in the tree recently, but we only need to bump the minor when we
    create a new minor branch. Some scripts also expect that the schema
    versions match between v2 and v2alpha1.

    Signed-off-by: Joe Stringer<EMAIL_ADDRESS>
 .github/maintainers-little-helper.yaml                   | 218 +++++++++++++++++++++-----------------------------------------
 CODEOWNERS                                               | 268 ++--------------------------------------------------------------------------
 Documentation/network/kubernetes/compatibility-table.rst |   6 +-
 pkg/k8s/apis/cilium.io/v2alpha1/register.go              |   2 +-
 stable.txt                                               |   1 -
 5 files changed, 86 insertions(+), 409 deletions(-)

v1.12 branch commit:
$ git show --stat 9b9caf949e3c88469b10bc0bd1c89603b00f762c
commit 9b9caf949e3c88469b10bc0bd1c89603b00f762c
Author: André Martins<EMAIL_ADDRESS>Date:   Tue Jun 21 23:49:32 2022 +0200

    Prepare v1.12 stable branch

    Signed-off-by: André Martins<EMAIL_ADDRESS>
 .github/ISSUE_TEMPLATE/bug_report.yaml                     |  83 --------------
 .github/ISSUE_TEMPLATE/failing_test_jenkins_template.yaml  | 115 -------------------
 .github/ISSUE_TEMPLATE/failing_test_template.md            |  16 ---
 .github/ISSUE_TEMPLATE/feature_template.md                 |  25 ----
 .github/dependabot.yml                                     | 460 --------------------------------------------------------------------------
 .github/maintainers-little-helper.yaml                     | 174 ++++++++++++----------------
 .github/pull_request_template.md                           |  18 ---
 .github/workflows/build-images-ci.yaml                     | 182 ++++-------------------------
 .github/workflows/build-images-hotfixes.yaml               |   2 +-
 .github/workflows/build-images-releases.yaml               |   4 +-
 .github/workflows/ci-images-cache-cleaner.yaml             |  64 -----------
 .github/workflows/ci-images-garbage-collect.yaml           |  23 ----
 .github/workflows/close-stale-issues.yml                   |  44 -------
 .github/workflows/conformance-aks-v1.10.yaml               | 324 ----------------------------------------------------
 .github/workflows/conformance-aks-v1.11.yaml               | 341 -------------------------------------------------------
 .github/workflows/conformance-aks.yaml                     | 346 -------------------------------------------------------
 .github/workflows/conformance-aws-cni-v1.10.yaml           | 356 ---------------------------------------------------------
 .github/workflows/conformance-aws-cni-v1.11.yaml           | 351 --------------------------------------------------------
 .github/workflows/conformance-aws-cni.yaml                 | 354 ---------------------------------------------------------
 .github/workflows/conformance-eks-v1.10.yaml               | 355 ---------------------------------------------------------
 .github/workflows/conformance-eks-v1.11.yaml               | 375 ------------------------------------------------------------
 .github/workflows/conformance-eks.yaml                     | 378 ------------------------------------------------------------
 .github/workflows/conformance-externalworkloads-v1.10.yaml | 351 --------------------------------------------------------
 .github/workflows/conformance-externalworkloads-v1.11.yaml | 362 ----------------------------------------------------------
 .github/workflows/conformance-externalworkloads.yaml       | 365 ----------------------------------------------------------
 .github/workflows/conformance-gke-v1.10.yaml               | 380 -------------------------------------------------------------
 .github/workflows/conformance-gke-v1.11.yaml               | 396 ---------------------------------------------------------------
 .github/workflows/conformance-gke.yaml                     | 400 ----------------------------------------------------------------
 .github/workflows/conformance-ingress.yaml                 | 165 ---------------------------
 .github/workflows/conformance-k8s-network-policies.yaml    | 137 ----------------------
 .github/workflows/conformance-kind.yaml                    |   4 +-
 .github/workflows/conformance-multicluster-v1.10.yaml      | 383 -------------------------------------------------------------
 .github/workflows/conformance-multicluster-v1.11.yaml      | 400 ----------------------------------------------------------------
 .github/workflows/conformance-multicluster.yaml            | 429 ---------------------------------------------------------------------
 .github/workflows/documentation.yaml                       |   4 +-
 .github/workflows/lint-bpf-checks.yaml                     |   4 +-
 .github/workflows/lint-codeowners.yaml                     | 128 ---------------------
 .github/workflows/lint-codeql.yaml                         |   8 +-
 .github/workflows/lint-go.yaml                             |   4 +-
 .github/workflows/lint-images-base.yaml                    |   4 +-
 .github/workflows/tests-cifuzz.yaml                        |  30 -----
 .github/workflows/tests-l4lb-v1.10.yaml                    | 249 ----------------------------------------
 .github/workflows/tests-l4lb-v1.11.yaml                    | 249 ----------------------------------------
 .github/workflows/tests-l4lb.yaml                          | 253 -----------------------------------------
 .github/workflows/tests-nightly.yaml                       |  69 -----------
 .github/workflows/tests-smoke-ipv6.yaml                    |   4 +-
 .github/workflows/tests-smoke.yaml                         |   4 +-
 install/kubernetes/Makefile.values                         |  16 +--
 install/kubernetes/cilium/Chart.yaml                       |   6 +-
 install/kubernetes/cilium/README.md                        |  24 ++--
 install/kubernetes/cilium/values.yaml                      |  40 +++----
 51 files changed, 156 insertions(+), 9102 deletions(-)

Does this PR need to create the v1.13 versions of the workflows in https://github.com/cilium/cilium/tree/master/.github/workflows, i.e. conformance-aks-v1.13.yaml and friends, or does that happen seperately?

@bimmlerd no, those go into the master branch, see here: https://github.com/cilium/cilium/pull/22614


Issue for executing KITTI bag files.

hello! i have an issue when executing KITTI open-datasets.
Do you have any advice for these errors?
[pose_graph_manager-5] [INFO] [1679576950.931437450] [r0.cslam_pose_graph_manager]: Visualization enabled.
[pose_graph_manager-5] [INFO] [1679576950.938801026] [r0.cslam_pose_graph_manager]: Simulated rendezvous schedule of robot 0,3500,10800
[pose_graph_manager-5] [INFO] [1679576950.938833371] [r0.cslam_pose_graph_manager]: Initialization done.
[map_manager-4] [INFO] [1679576951.113992534] [r0.cslam_map_manager]: Initialization done.
[loop_closure_detection_node.py-3] [INFO] [1679576952.702047141] [r0.cslam_loop_closure_detection]: Using CosPlace. (default)
[loop_closure_detection_node.py-3] [INFO] [1679576953.389402179] [r0.cslam_loop_closure_detection]: loading checkpoint '/home/user/Swarm-SLAM/install/cslam/share/cslam/models/resnet101_512.pth'
[loop_closure_detection_node.py-3] /home/user/.local/lib/python3.8/site-packages/torchvision/models/_utils.py:208: UserWarning: The parameter 'pretrained' is deprecated since 0.13 and may be removed in the future, please use 'weights' instead.
[loop_closure_detection_node.py-3]   warnings.warn(
[loop_closure_detection_node.py-3] /home/user/.local/lib/python3.8/site-packages/torchvision/models/_utils.py:223: UserWarning: Arguments other than a weight enum or `None` for 'weights' are deprecated since 0.13 and may be removed in the future. The current behavior is equivalent to passing `weights=ResNet101_Weights.IMAGENET1K_V1`. You can also use `weights=ResNet101_Weights.DEFAULT` to get the most up-to-date weights.
[loop_closure_detection_node.py-3]   warnings.warn(msg)
[loop_closure_detection_node.py-3] /home/user/.local/lib/python3.8/site-packages/torchvision/transforms/transforms.py:329: UserWarning: Argument 'interpolation' of type int is deprecated since 0.13 and will be removed in 0.15. Please use InterpolationMode enum.
[loop_closure_detection_node.py-3]   warnings.warn(
[loop_closure_detection_node.py-3] Traceback (most recent call last):
[loop_closure_detection_node.py-3]   File "/opt/ros/foxy/lib/python3.8/site-packages/rosidl_generator_py/import_type_support_impl.py", line 46, in import_type_support
[loop_closure_detection_node.py-3]     return importlib.import_module(module_name, package=pkg_name)
[loop_closure_detection_node.py-3]   File "/home/user/anaconda3/envs/cslam2/lib/python3.8/importlib/__init__.py", line 127, in import_module
[loop_closure_detection_node.py-3]     return _bootstrap._gcd_import(name[level:], package, level)
[loop_closure_detection_node.py-3]   File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
[loop_closure_detection_node.py-3]   File "<frozen importlib._bootstrap>", line 991, in _find_and_load
[loop_closure_detection_node.py-3]   File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
[loop_closure_detection_node.py-3] ModuleNotFoundError: No module named 'cslam_common_interfaces.cslam_common_interfaces_s__rosidl_typesupport_c'
[loop_closure_detection_node.py-3]
[loop_closure_detection_node.py-3] During handling of the above exception, another exception occurred:
[loop_closure_detection_node.py-3]
[loop_closure_detection_node.py-3] Traceback (most recent call last):
[loop_closure_detection_node.py-3]   File "/home/user/Swarm-SLAM/install/cslam/lib/cslam/loop_closure_detection_node.py", line 107, in <module>
[loop_closure_detection_node.py-3]     lcd = LoopClosureDetection()
[loop_closure_detection_node.py-3]   File "/home/user/Swarm-SLAM/install/cslam/lib/cslam/loop_closure_detection_node.py", line 97, in __init__
[loop_closure_detection_node.py-3]     self.glcd = GlobalDescriptorLoopClosureDetection(
[loop_closure_detection_node.py-3]   File "/home/user/Swarm-SLAM/install/cslam/lib/python3.9/site-packages/cslam/global_descriptor_loop_closure_detection.py", line 66, in __init__
[loop_closure_detection_node.py-3]     self.global_descriptor_publisher = self.node.create_publisher(
[loop_closure_detection_node.py-3]   File "/opt/ros/foxy/lib/python3.8/site-packages/rclpy/node.py", line 1144, in create_publisher
[loop_closure_detection_node.py-3]     check_for_type_support(msg_type)
[loop_closure_detection_node.py-3]   File "/opt/ros/foxy/lib/python3.8/site-packages/rclpy/type_support.py", line 29, in check_for_type_support
[loop_closure_detection_node.py-3]     msg_type.__class__.__import_type_support__()
[loop_closure_detection_node.py-3]   File "/home/user/Swarm-SLAM/install/cslam_common_interfaces/lib/python3.9/site-packages/cslam_common_interfaces/msg/_global_descriptors.py", line 27, in __import_type_support__
[loop_closure_detection_node.py-3]     module = import_type_support('cslam_common_interfaces')
[loop_closure_detection_node.py-3]   File "/opt/ros/foxy/lib/python3.8/site-packages/rosidl_generator_py/import_type_support_impl.py", line 48, in import_type_support
[loop_closure_detection_node.py-3]     raise UnsupportedTypeSupport(pkg_name)
[loop_closure_detection_node.py-3] rosidl_generator_py.import_type_support_impl.UnsupportedTypeSupport: Could not import 'rosidl_typesupport_c' for package 'cslam_common_interfaces'
[ERROR] [loop_closure_detection_node.py-3]: process has died [pid 2343015, exit code 1, cmd '/home/user/Swarm-SLAM/install/cslam/lib/cslam/loop_closure_detection_node.py --ros-args -r __node:=cslam_loop_closure_detection -r __ns:=/r0 --params-file /home/user/Swarm-SLAM/install/cslam_experiments/share/cslam_experiments/config/kitti_stereo.yaml --params-file /tmp/launch_params_2d376iss'].
[INFO] [bag-7]: process started with pid [2343315]
[bag-7] [ERROR] [1679576976.225999835] [rosbag2_storage]: Could not open '/home/user/Swarm-SLAM/install/cslam_experiments/share/cslam_experiments/data/KITTI360-09_5robots/KITTI360-09-0/kitti_2011_09_26_drive_0009_synced.bag' with 'sqlite3'. Error: Failed to setup storage. Error: Error when processing SQL statement. SQLite error (26): file is not a database
[bag-7] [ERROR] [1679576976.226025581] [rosbag2_storage]: Could not load/open plugin with storage id 'sqlite3'.
[bag-7] [ERROR] [1679576976.226039745] [rosbag2_transport]: Failed to play: No storage could be initialized. Abort
[INFO] [bag-7]: process has finished cleanly [pid 2343315]

It seems that the cslam_common_interfaces package is not found.

Make sure the cslam_common_interfaces package is in your src folder
Try rm -r build install log and rebuild.
Make sure to source your workspace before launching : source install/setup.bash

As for the bag error, it seems that you are running a ROS 1 bag instead of a ROS 2 one. So you either need to convert it from ROS 1 to ROS 2 or run the ros1_bridge in parallel


Get PHOTO catalog up and running for comparisons

Nice results.
Any idea what is happening with the 22 mag galaxies? It may be worthwhile to spot check a few of the 22 magnitude galaxies that bliss misses but that Photo detects. It seems fishy thay bliss would do better at all the magnitudes but this one

Before adjusting for the training Poisson rate in #512, our recall was much higher across the board (at the cost of precision):
detections
           recall  expected_recall  precision  expected_precision    n  n_galaxies  n_galaxies_detected
18       0.968750              NaN   1.000000                 NaN   32          10                    9
19       0.875000              NaN   0.853659                 NaN   56          29                   23
20       0.835294              NaN   0.823529                 NaN   85          59                   46
21       0.886364              NaN   0.746269                 NaN  176         132                  112
22       0.682008              NaN   0.380198                 NaN  239         158                   84
23       0.486726              NaN   0.500000                 NaN  113          47                    6
overall  0.761317         0.537483   0.594800            0.847694  729         445                  300

So it could simply be a matter of PHOTO being more sensitive.

That could be it. There may also be issues with the "ground truth" (coadd). To get a feel for how reliable our ground truth is, you might sample 10 or 20 objects where Photo and BLISS differ (either at detection or classification) and, for these objects, assess whether the coadd "ground truth" results agree with the DECaLs catalog. DeCALs results can be looked up here: https://www.legacysurvey.org/viewer#IC 2038
Neither coadd nor DeCALS is totally authoritative. Both should be a lot more accurate than results computed with a single frame. If coadd and DeCALs both agree about a particular detection or classification, despite being quite different algorithmically, and using totally different image data, we can have high confidence in our "ground truth."

Here's an illustration of how we're misspecifying the distribution flux. I show the current pareto distribution used in BLISS alongside a pareto fit to all the galaxies in COADD ("Fitted Pareto"). I then show the COADD Pareto distribution truncated at the same lower bound we're currently using. This shows that BLISS is sampling roughly twice as many galaxies around 1000 flux (~22.3 magnitude) as show up in COADD. This could explain why there is such a precision drop around 21-22 magnitude.


Fix #24 - Limit the extensibility of classes and methods

Apply Guideline 4-5 / EXTEND-5: Limit the extensibility of classes and
methods from the Secure Coding Guidelines for Java SE
https://www.oracle.com/technetwork/java/seccodeguide-139067.html#4-5
(reported by running VisualCodeGrepper
https://github.com/nccgroup/VCG/blob/master/VisualCodeGrepper/modJavaCheck.vb#L318
)

Thanks for the PR - I'm not 100% this is needed/beneficial. Whilst it does improve security and composition should be favoured over inheritence generally, is it being too restrictive by not allowing consumers to override RecoveryCodeGenerator for example?

@samdjstevens honestly, I'm not sure either I should have made all classes final.
As stated in the Secure Coding Guidelines, we have two choices "Design classes and methods for inheritance or declare them final"...
So I have absolutely no problem in reverting some of them !!
I may have been a little overzealous :p

PR updated to revert use of final keyword for all classes which could be subclassed.


Animated gradients in CSS using interpolations and JavaScript Animation.

Create custom animated gradients that can have color values generated on-the-fly or different timing functions,eases as well different color spaces.  Should probably take a Map that has all the customizable options.

This seems to be a bit out of the project scope for now. Maybe someday....


Feature Request: Use a local file as background image for transulency

Feature Requested
I'd like to be able to use an image file inside the vault as a background image - currently it seems only web accessible images via URL can be added currently.
Checklist

[x] The feature would be useful to more users than just me. (→ If it's only relevant to you, have you considered using a CSS Snippet?)

Looks like Obsidian disabled this recently, closing!


Remove Ansible variables

With the move to terraform, we can remove many of the Ansible variable flies.
Looking at the inventory/ directory, it seems that most can go.
Let's assume that the plan is to delete them all. What needs to remain?
These seem ready to remove:
inventory/1-datacenter
inventory/2-datacenter

Remove:
inventory/group_vars/dc1
inventory/group_vars/dc2

I think we need to move these to terraform (if not there already):
inventory/group_vars/consul_clients
inventory/group_vars/mesos_leaders

Not sure what this is used for, it just contains a .gitkeep:
inventory/host_vars

Finally, all.yml can probably have the openstack vars removed. Can we remove
ansible_ssh_user: centos
ansible_ssh_key: ansible_key
ansible_ssh_pubkey: "{{ lookup('file','~/.ssh/id_rsa.pub') }}"

And
consul_servers_group: consul_servers
consul_dns_domain: consul

Marking as closed. /inventory has been removed as of 0.3.0


fix(DefinitionResolver): find variables in sibling children

Fixes #566

Thanks, this is a better solution that mine, I was struggling to work out how to do something like this as your suggestion.

This also includes a change unrelated to the bug, in that before we only checked previous siblings and parents, but not the descendants of those. So this wouldn't register $foo before:
if (true) {
  $foo = 123
}
$f| // no completion for $foo

now it should.
With that change I could make it so that the search for the foreach key/value starts at the parent/sibling too because it can then traverse into the ForeachStatement, but not if that was the starting point.

@jens1o @phil-nelson if anyone wants to test: php-intellisense-1.5.4.vsix.zip

I've had a play with this branch, no problems so far :thumbsup:

Works liek a charm :)


AEROGEAR-8514 - Fix unique client and users metrics

Fixes the problem mentioned at https://github.com/aerogear/voyager-server/pull/109
Verification instructions:

Go to this URL
https://prometheus-aaa.apps.aliok-6268.openshiftworkshop.com/graph?g0.range_input=30m&g0.end_input=2019-02-19 09%3A11&g0.expr=count( count by (client_id) (clients) )&g0.tab=0

See the values in the graph.

You can also try count( count by (user_id) (users) ) but there is only 1 user in the data I generated.

Mr @psturc  can you have a look please?

👀


Incompatibilities with current Rails versions.

Once added the line to Rails 5 application's Gemfile and trying to bundle install I get a number of error messages that seem to point to versions conflicts:
silverdr$ bundle install
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies...
Bundler could not find compatible versions for gem "rack":
  In snapshot (Gemfile.lock):
    rack (= 2.0.3)

  In Gemfile:
    rails-controller-testing was resolved to 1.0.2, which depends on
      actionpack (>= 5.0.1, ~> 5.x) was resolved to 5.0.5, which depends on
        rack (~> 2.0)

    bitpay-sdk was resolved to 2.1.0, which depends on
      rack (~> 1.5.2)

    rails-controller-testing was resolved to 1.0.2, which depends on
      actionpack (>= 5.0.1, ~> 5.x) was resolved to 5.0.5, which depends on
        rack-test (~> 0.6.3) was resolved to 0.6.3, which depends on
          rack (>= 1.0)

    sass-rails (~> 5.0) was resolved to 5.0.6, which depends on
      sprockets (< 4.0, >= 2.8) was resolved to 3.7.1, which depends on
        rack (< 3, > 1)

Running `bundle update` will rebuild your snapshot from scratch, using only
the gems in your Gemfile, which may resolve the conflict.

Running bundle update doesn't help either - generates even more error messages:
$ bundle update
The dependency tzinfo-data (>= 0) will be unused by any of the platforms Bundler is installing for. Bundler is installing for ruby but the dependency is only for x86-mingw32, x86-mswin32, x64-mingw32, java. To add those platforms to the bundle, run `bundle lock --add-platform x86-mingw32 x86-mswin32 x64-mingw32 java`.
Fetching gem metadata from https://rubygems.org/...........
Fetching version metadata from https://rubygems.org/...
Fetching dependency metadata from https://rubygems.org/..
Resolving dependencies.......................................
Bundler could not find compatible versions for gem "actionpack":
  In Gemfile:
    rails (~> 5.0.0) was resolved to 5.0.0, which depends on
      actionpack (= 5.0.0)

    rails (~> 5.0.0) was resolved to 5.0.0, which depends on
      actionpack (= 5.0.0)

    rails (~> 5.0.0) was resolved to 5.0.0, which depends on
      actionpack (= 5.0.0)

    rails (~> 5.0.0) was resolved to 5.0.0, which depends on
      actionpack (= 5.0.0)

    rails-controller-testing was resolved to 1.0.2, which depends on
      actionpack (>= 5.0.1, ~> 5.x)

    sass-rails (~> 5.0) was resolved to 5.0.6, which depends on
      sprockets-rails (< 4.0, >= 2.0) was resolved to 3.2.0, which depends on
        actionpack (>= 4.0)
Bundler could not find compatible versions for gem "rack":
  In Gemfile:
    rails (~> 5.0.0) was resolved to 5.0.0, which depends on
      actionpack (= 5.0.0) was resolved to 5.0.0, which depends on
        rack (~> 2.0)

    bitpay-sdk was resolved to 2.1.0, which depends on
      rack (~> 1.5.2)

    rails (~> 5.0.0) was resolved to 5.0.0, which depends on
      actionpack (= 5.0.0) was resolved to 5.0.0, which depends on
        rack-test (~> 0.6.3) was resolved to 0.6.3, which depends on
          rack (>= 1.0)

    sass-rails (~> 5.0) was resolved to 5.0.6, which depends on
      sprockets (< 4.0, >= 2.8) was resolved to 3.7.1, which depends on
        rack (< 3, > 1)

Once the Gemfile line
gem 'bitpay-sdk', :require  => 'bitpay_sdk'

is removed or commented out, there are no errors at all.

@silverdr did you see the discussion in https://github.com/bitpay/ruby-client/pull/56? -> using the master of the gem could help.

@rakvat no, I didn't - thanks. But still - that PR seems to be open for many months now so I am hesitant to go further with this. I simply don't feel convinced that I can safely recommend it to my customers if things, especially on the financial side of the applications, require some kind of unofficial workarounds.


UT for victims package

Add unit tests for victims package - #41

Hey. I'm sorry, but you might have misunderstood my last comment. I'm totally ok with using the assert package  - in fact I prefer we use that. But we would need to add it to glide.yaml. If you could revert your last commit and add it to the glide file, this PR is perfect.

You would just need to add this
diff --git a/glide.yaml b/glide.yaml
index 2c12b908..639e8600 100644
--- a/glide.yaml
+++ b/glide.yaml
@@ -19,3 +19,4 @@ import:
   subpackages:
   - kubernetes
   - rest
+- package: github.com/stretchr/testify

I went ahead and merged this PR. I revert your last commit, and added the dependency to github.com/stretchr/testify`.
Once again, thanks for this.


Test of full background vs. approximate background in PhoSim

This is carried over from issue #19:
I would like to suggest a few limited tests on one FOV (or some reasonable fraction thereof), where we compare the full-on background, the approximate background, and runs with fixed sources including all the background options. This way, we will have a data set where at least some of these questions can be directly addressed quantitatively.
Some tests have been done but it would be good to do some more targeted tests. @fjaviersanchez We should add that to your list of validations, please. Thanks!

The results for these tests have been recorded in issue #140 so I close this issue.


uint8array-json-parser

https://www.npmjs.com/package/uint8array-json-parser
Hi, sorry to bother you about this old code, it works fine (and it is the only solution I found to load a large json file on 32bit browsers when it isn't allowed to cut the json in smaller pieces) but it may be a bit too slow (mine is >300MB, once compressed it is 25MB).
Do you think we should go directly ask V8 for a native implementation? (it should be just a cast of the pointer used in JSON.parse)
Or maybe they prefer to parse a string because it is immutable? If so is there any solution?

Note to anyone else reading this: this is the wrong repo. The repo is here instead: https://github.com/evanw/uint8array-json-parser.

Do you think we should go directly ask V8 for a native implementation?

You are welcome to. I'm not connected at all with the V8 team though, so I'm not going to be able to help you. I assume the best avenues for doing this are asking for an API from node instead if you only need it in node, or trying to advocate for getting it into the JavaScript language standard if you need it in the browser. The node route seems much easier. Adding an API to JavaScript is hard but not impossible. You'd need to advocate for it yourself though. Instructions for how to do that are here.


[FLINK-31424][table-planner] Fix NPE produced by multiple sink with local-global window aggregate

What is the purpose of the change
This pull request fixes the NPE when getting RelWindowProperties for StreamPhysicalGlobalWindowAggregate.  This is caused by when optimized the following DAG, where the WindowJoin node is the breaking point.

Brief change log

Let FlinkRelMdWindowProperties support computing window properties for StreamPhysicalGlobalWindowAggregate.

Verifying this change
This change added tests and can be verified as follows:

Add FlinkRelMdWindowPropertiesTest to verify the window properties computation on different rel nodes.
Add test underWindowJoinTest to verify the case.

Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

@flinkbot run azure

@LadyForest could you also create a backport pr to release-1.17?

@LadyForest could you also create a backport pr to release-1.17?

Sure :)


"Introducing fastpages" Blog Post

in addition to this blog post, made the following minor corrections

corrected css to include list elements
correcting a minor spacing issue in the site header
added instructions on removing example blog posts.
make sure this blog post gets deleted in the setup process

Note: once this PR is merged, it will take ~ 5 minutes for the blog to appear on https://fastpages.fast.ai/

/preview

/preview

/preview

@jph00 the launch blog post is ready for review.  A preview is available here:  https://5e5078158b8469a184037ad9--fastpages.netlify.com/2020/02/21/introducing-fastpages.html
(I suppose we can use the NBReview App if we have comments on the post)

/preview

Generating a fresh preview, as I merged a bunch of PRs while this was opened and since merged those changes back into this branch.
/preview

Looking better @hamelsmu . Here's a suggested redraft of the initial features list:

Create posts containing code, code outputs (which can be interactive), formatted text, and so forth, directly from Jupyter Notebooks; for instance see this great example post from Scott Hawley. Notebook posts support features such as:

Hide or show cell input and output.
Collapsable code cells that are either open or closed by default.
Define the Title, Summary and other metadata via a special markdown cells
Add links to Colab and GitHub automatically

Create posts, including formatting and images, directly from Microsoft Word documents
Create and edit Markdown posts entirely online using GitHub's built-in markdown editor
Embed Twitter cards and YouTube videos
... and much more.

Then you can also remove the stuff about online markdown editor and the link to Scott's blog.
Suggest waiting until Monday to launch since weekends are quiet.

Sounds good, @jph00.  Summary of changes I just made:

Changed the feature list per the above feedback
Added documentation re: the new categories feature to the notebook
removed redundant mention of Scott's blog from the end b/c now its in the feature bulleted list
removed mention of online markdown editor b/c it is now in the bulleted feature list

/preview

/preview

/preview

Ok image previews are supported (tested this in slack message to myself)

/preview

Merging this now @jph00 will let you do the honors of announcing it whenever you are ready (will take ~ 5 min for it to show up )