[TCD-2485] Clarify how to update RDC jobs

Description
https://saucedev.atlassian.net/browse/TCD-2485
Motivation and Context
Clarifies that the update_job API method does not work for RDC - the Selenium JS Executor should be used instead. Also adds keywords to the Test Annotation doc to enhance SEO.
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Documentation fix (typos, incorrect content, missing content, etc.)

Checklist

[ ] I have read the contributing document.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

Thanks @wswebcreation!


Galactic + Protobuf typesupport: Runtime crash "File already exists in database"

ROS2 Galactic
Ubuntu 20.04
rmw_ecal master
rosidl_typesupport_protobuf master
ecal v5.10.0-alpha-10-g221e5b7e (11.03.2022)
$ protoc --version
libprotoc 3.17.1

When I run simple examples, I see something like this and the process ends:
[INFO] [talker]: process started with pid [2977830]
[talker] [libprotobuf ERROR google/protobuf/descriptor_database.cc:641] File already exists in database: builtin_interfaces/msg/Duration.proto
[talker] [libprotobuf FATAL google/protobuf/descriptor.cc:1371] CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):
[talker] terminate called after throwing an instance of 'google::protobuf::FatalException'
[talker]   what():  CHECK failed: GeneratedDatabase()->Add(encoded_file_descriptor, size):

Moved issue to typesupport repo, since its typesupport related issue


How can I start bash in ~ rather than in /mnt/c/Users/Jacob?

This bug-tracker is monitored by developers and other technical types.  We like detail!  So please use this form and tell us, concisely but precisely, what's up.  Please fill out ALL THE FIELDS!
If you have a feature request, please post to the UserVoice.  If you're reporting a BSOD, don't post here!  Instead, e-mail<EMAIL_ADDRESS>and if possible attach the minidump from "C:\Windows\minidump".

Your Windows build number:  (Type ver at a Windows Command Prompt) 15063.413

What you're doing and what's happening:  (Copy&paste specific commands and their output, or include screen shots) In command prompt, enter "bash". Starts bash at /mnt/c/Users/Jacob.

What's wrong / what should be happening instead: Would prefer to start in ~ (/home/jacob)

Strace of the failing command, if applicable:  (If <cmd> is failing, then run strace -o strace.txt -ff <cmd>, and post the strace.txt output here) N/A

See our contributing instructions for assistance.

More detail: When running "bash" directly, as opposed to starting it from CMD, it starts in ~ as I'd prefer. I've discovered that if I launch bash with the flag --login, it starts me at /mnt/c/Users/Jacob, so I suspect when I'm launching bash from command prompt that it's somehow passing that flag by default. Is there a way to negate that?

@JacobDB - I guess it's "bash ~" what you want  :)

Ah, that's it, thanks! I probably should have tried that, haha.

You can also

Create a shortcut to BASH on the taskbar (or wherever) and alter the properties to either run "bash ANYDIRYOUWANT"

Or simply edit the short cut's "start in" properties to make it point to whatever folder you wish

Or click in the "folder/path" bar area in Windows Explorer and then type "bash" to have Windows launch bash in the folder currently displayed in Explorer.

Finally, there are a few other ways but you can also edit your .bashrc and simply get it to change to the start folder of your choice.  :)


Content Improvements to the Streamlit App

Only modify the app.py for editing the content and its presentation on multiple pages

I would like to work on this issue, @smaranjitghose kindly assign it to me!


substring function requires update

For the Substring function, the third parameter (length) is an optional parameter. The "Otherwise" portion of the NOTE following that function should indicate that if the length parameter is not provided, then the Substring function will take all characters from the startIndex to the end of the string.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 03aa9e88-7fa0-56ab-f794-58f0c37b6d7a
Version Independent ID: 25c5ef5c-6ba4-f5ff-f94b-8f2818ec2feb
Content: Reference guide for functions in expressions - Azure Logic Apps
Content Source: articles/logic-apps/workflow-definition-language-functions-reference.md
Service: logic-apps
GitHub Login: @ecfan
Microsoft Alias: estfan

@eccountable Thank you for your feedback! We will review and update as appropriate.

@eccountable Thanks for reporting this item! I have updated the doc and it will be published soon. We appreciate the feedback!


Incorrect handle to frozenlists in Dataset object

What happened?
Dataset object could not be fetched with Dataset[variables] when variables is a frozenlist. There is some source code that can replicate this behaviour.
What did you expect to happen?
Dataset[variables] returning a new fetched dataset. It returns a key error.
Minimal Complete Verifiable Example
# %%
from dagster import Output, asset, Field, build_op_context
import numpy as np
import pandas as pd
import xarray as xr

def retrieve_dataset(date, fct_step, lats, lons, runtime, source, variables):

    dt = str(date)[0:10].replace("-", "")
    year = dt[0:4]
    prev_n = "{}".format(fct_step).zfill(3)

    lat_toplot = np.arange(
        min(lats), max(lats), 0.25
    )  # last number is exclusive
    lon_toplot = (
        np.arange(min(lons), max(lons), 0.25) % 360
    )  # last number is exclusive

    gfs_model = "ds084.1"
    URL = f"https://rda.ucar.edu/thredds/dodsC/files/g/{gfs_model}/{year}/{dt}/gfs.0p25.{dt}{runtime}.f{prev_n}.grib2"

    dataset = xr.open_dataset(URL, cache=True, engine="netcdf4")[variables]

    # Select required data via xarray
    dataset = dataset.sel(lon=lon_toplot, lat=lat_toplot)[variables]

    return dataset

@asset(
    description="GFS dataset fetched for NOAA Forecast data from NOAA NOMADS server.",
    config_schema=dict(
        variables=Field(list),
    ),
    output_required=True,
)
def ncar_dataset(context):

    date = "2022-02-01"
    fct_step = 0
    lats = [-4.0, -2.75]
    lons = [-41.5, -40.25]
    runtime = "00"
    # variables = list(context.op_config["variables"])  <--- This way it runs
    variables = context.op_config["variables"]
    source = "model-gfs-003"

    dataset = retrieve_dataset(
        date, fct_step, lats, lons, runtime, source, variables
    )

    return Output(dataset)

test_variable = "u-component_of_wind_height_above_ground"

with build_op_context(
    op_config={
        "variables": [test_variable],
    }
) as context:

    dataset_ast = ncar_dataset(context)
dataset = dataset_ast.value

MVCE confirmation

[X] Minimal example — the example is as focused as reasonably possible to demonstrate the underlying issue in xarray.
[ ] Complete example — the example is self-contained, including all data and the text of any traceback.
[X] Verifiable example — the example copy & pastes into an IPython prompt or Binder notebook, returning the result.
[X] New issue — a search of GitHub Issues suggests this is not a duplicate.

Relevant log output
I raises KeyError

Anything else we need to know?
No response
Environment

Python 3.10.4
pandas==1.5.0
xarray==0.19.0
netCDF4==1.5.7
dagster
dagit

Hi @airton-neto - here is a much shorter example that reproduces the same error
url = 'https://rda.ucar.edu/thredds/dodsC/files/g/ds084.1/2022/20220201/gfs.0p25.2022020100.f000.grib2'

dataset = xr.open_dataset(url, cache=True, engine="netcdf4")

from dagster._utils import frozenlist

# variables = list(['u-component_of_wind_height_above_ground']) #  <--- This way it runs
variables = frozenlist(['u-component_of_wind_height_above_ground'])

dataset[variables]

I'm not immediately sure why this happens, but the frozenlist class is a dagster private internal, and the problem is specific to how that object interacts with the current xarray code. As a downstream user of xarray, can dagster not just change it's code to pass the type xarray expects (i.e. a normal list)?
Having said that if you want to submit a PR which fixes this bug (and doesn't require special-casing dagster somehow) then that would be welcome too!

Hey @TomNicholas, thanks for replying me back!
The problem is they chose to define how to hash a list (which is no hashable) by parsing it to a tuple, which is the frozenlist
class frozenlist(list):
    def __readonly__(self, *args, **kwargs):
        raise RuntimeError("Cannot modify ReadOnlyList")

    # https://docs.python.org/3/library/pickle.html#object.__reduce__
    #
    # Like frozendict, implement __reduce__ and __setstate__ to handle pickling.
    # Otherwise, __setstate__ will be called to restore the frozenlist, causing
    # a RuntimeError because frozenlist is not mutable.

    def __reduce__(self):
        return (frozenlist, (), list(self))

    def __setstate__(self, state):
        self.__init__(state)

    __setitem__ = __readonly__  # type: ignore[assignment]
    __delitem__ = __readonly__
    append = __readonly__
    clear = __readonly__
    extend = __readonly__
    insert = __readonly__
    pop = __readonly__
    remove = __readonly__
    reverse = __readonly__
    sort = __readonly__  # type: ignore[assignment]

    def __hash__(self):
        return hash(tuple(self))

Dataset class defines getitem by
    def __getitem__(
        self: T_Dataset, key: Mapping[Any, Any] | Hashable | Iterable[Hashable]
    ) -> T_Dataset | DataArray:
        """Access variables or coordinates of this dataset as a
        :py:class:`~xarray.DataArray` or a subset of variables or a indexed dataset.
        Indexing with a list of names will return a new ``Dataset`` object.
        """
        if utils.is_dict_like(key):
            return self.isel(**key)
        if utils.hashable(key):
            return self._construct_dataarray(key)
        if utils.iterable_of_hashable(key):
            return self._copy_listed(key)
        raise ValueError(f"Unsupported key-type {type(key)}")

It tries to hash the key (and this frozenlist do this without any error).
I've checked out how pandas library deal with this task. They have a much more complex structure to deal with this problem (I haven't read all of the code below to understand):
    def __getitem__(self, key):
        check_deprecated_indexers(key)
        key = lib.item_from_zerodim(key)
        key = com.apply_if_callable(key, self)

        if is_hashable(key) and not is_iterator(key):
            # is_iterator to exclude generator e.g. test_getitem_listlike
            # shortcut if the key is in columns
            is_mi = isinstance(self.columns, MultiIndex)
            # GH#45316 Return view if key is not duplicated
            # Only use drop_duplicates with duplicates for performance
            if not is_mi and (
                self.columns.is_unique
                and key in self.columns
                or key in self.columns.drop_duplicates(keep=False)
            ):
                return self._get_item_cache(key)

            elif is_mi and self.columns.is_unique and key in self.columns:
                return self._getitem_multilevel(key)
        # Do we have a slicer (on rows)?
        indexer = convert_to_index_sliceable(self, key)
        if indexer is not None:
            if isinstance(indexer, np.ndarray):
                indexer = lib.maybe_indices_to_slice(
                    indexer.astype(np.intp, copy=False), len(self)
                )
                if isinstance(indexer, np.ndarray):
                    # GH#43223 If we can not convert, use take
                    return self.take(indexer, axis=0)
            # either we have a slice or we have a string that can be converted
            #  to a slice for partial-string date indexing
            return self._slice(indexer, axis=0)

        # Do we have a (boolean) DataFrame?
        if isinstance(key, DataFrame):
            return self.where(key)

        # Do we have a (boolean) 1d indexer?
        if com.is_bool_indexer(key):
            return self._getitem_bool_array(key)

        # We are left with two options: a single key, and a collection of keys,
        # We interpret tuples as collections only for non-MultiIndex
        is_single_key = isinstance(key, tuple) or not is_list_like(key)

        if is_single_key:
            if self.columns.nlevels > 1:
                return self._getitem_multilevel(key)
            indexer = self.columns.get_loc(key)
            if is_integer(indexer):
                indexer = [indexer]
        else:
            if is_iterator(key):
                key = list(key)
            indexer = self.columns._get_indexer_strict(key, "columns")[1]

        # take() does not accept boolean indexers
        if getattr(indexer, "dtype", None) == bool:
            indexer = np.where(indexer)[0]

        data = self._take_with_is_copy(indexer, axis=1)

        if is_single_key:
            # What does looking for a single key in a non-unique index return?
            # The behavior is inconsistent. It returns a Series, except when
            # - the key itself is repeated (test on data.shape, #9519), or
            # - we have a MultiIndex on columns (test on self.columns, #21309)
            if data.shape[1] == 1 and not isinstance(self.columns, MultiIndex):
                # GH#26490 using data[key] can cause RecursionError
                return data._get_item_cache(key)

        return data

This similar code, by the way, runs correctly
# %%
import pandas as pd
data = pd.DataFrame({'a': [1], 'b': [2]})
variables = frozenlist(['a'])
data[variables]

Now it is a matter of how xarray want to treat this issue.
If you do want to change your way of doing this, I could try coding and request this PR trying pandas approach.
Anyway, I will report this issue back to Dagster for them to check this discussion too. Maybe they see their side is worth changing.
Let me know what is your view about this!
Links:
https://github.com/pydata/xarray/blob/e1936a98059ae29da2861f58a7aff4a56302aac1/xarray/core/dataset.py#L1419
https://github.com/pandas-dev/pandas/blob/v1.5.1/pandas/core/frame.py#L473-L11983

I think we wanted to explicitly support any hashable (e.g. tuples or as in this case frozenlists) as variable or dimension "names".

I see, the user would be responsible to manage whats he's doing.
I think this answer closes this issue, then, as this behaviour is indeed expected.
Thanks for your helping. Nice work!


ClientNotFoundError: 404 Client Error for challenge

Hi I'm running the medias.py test with my account and I can't seem to get it work?
Running this on terminal on mac. username & passwords are correct. I don't know where to get the cookie...?
I'm getting this error:
2021-03-14 15:37:07,173 INFO public_request: _kyanrehorst_ [404] POST https://i.instagram.com/api/challenge/7182357309/K2scazkHeQ/ (<IP_ADDRESS>.135, Xiaomi capricorn)
2021-03-14 15:37:07,173 WARNING instagrapi: Status 404: Endpoint /challenge/7182357309/K2scazkHeQ/ does not exists
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/private.py", line 173, in _send_private_request
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: https://i.instagram.com/api/v1/accounts/login/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/private.py", line 293, in private_request
    self._send_private_request(endpoint, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/private.py", line 203, in _send_private_request
    raise ChallengeRequired(**last_json)
instagrapi.exceptions.ChallengeRequired: challenge_required

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/private.py", line 173, in _send_private_request
    response.raise_for_status()
  File "/usr/local/lib/python3.9/site-packages/requests/models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: https://i.instagram.com/api/challenge/7182357309/K2scazkHeQ/

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/quinten/instagrapi/examples/medias.py", line 112, in <module>
    cl.login(IG_USERNAME, IG_PASSWORD)
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/auth.py", line 341, in login
    if self.private_request("accounts/login/", data, login=True):
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/private.py", line 304, in private_request
    self.challenge_resolve(self.last_json)
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/challenge.py", line 68, in challenge_resolve
    return self.challenge_resolve_simple(challenge_url)
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/challenge.py", line 372, in challenge_resolve_simple
    self._send_private_request(challenge_url, {"choice": CHOICE_EMAIL})
  File "/usr/local/lib/python3.9/site-packages/instagrapi/mixins/private.py", line 239, in _send_private_request
    raise ClientNotFoundError(e, response=e.response, **last_json)
instagrapi.exceptions.ClientNotFoundError: 404 Client Error: Not Found for url: https://i.instagram.com/api/challenge/7182357309/K2scazkHeQ/

@quintendewilde I'm rewrite the mechanism for challenge resolving now, most likely this will help you

Fixed, use instagrapi==1.11.1


Use ENV var to debug/see raw excon requests.

It would be nice to see the raw request Excon assembles and sends
down the pipe. I wouldn't think there should be a performance hit granted all it's doing
is checking for a EXCON_DEBUG_REQUESTS environment variable.

I totally agree with the intent of this patch, but implementation-wise I'd shy away from a forced dump to $stderr. It'd be nice if in configuration/instantiation you could assign a logger object, which would default to nil.
If assigned excon could just call #debug on that object and it would be immediately compatible with the std-lib Logger as well as nice full-featured alternatives like logging and any other wrappers that someone might want to create, including a quick one-off which would dump to $stderr.puts.

How about doing something that integrates with instrumentation?

@dpiddy @nextmat Well, I basically want a quick and dirty way to see the raw request. Instrumentation is almost too much hassle to just run some fog stuff from a given console and want to see the raw request during runtime. Also I just realized that the debug condition is done of instantiation. Actually the more I think of it what I want for was basically to do something like
Excon.debug = "whatevs"
during runtime and then see the raw requests, and be able to shut it later:
Excon.debug = nil
Thoughts?

@shaiguitar - hmm.  Not sure how to best support this.  Does it really need explicitly to be the raw request or would the hashes be good enough?
I would kind of like to utilize the instrumentation code for things of this nature, rather than adding an additional path for doing something quite similar.  That said, I agree that it is kind of a hassle currently to instantiate/setup everything for a one-off debug foray.  Maybe we should add some kind of simple instrumentor you could easily turn on that would just dump to stdout and/or stderr?  Seems like that might keep the code clean but add a lot of value.  Thoughts?

@dpiddy @geemus -
OK, so if we go by doing it via the instrumentor we'd change those debug lines in this pull request to have something like this right:
`
if params.has_key?(:instrumentor)
params[:instrumentor].instrument("#{params[:instrumentor_name]}.raw_request", :raw_request => request)
end
`
That way we don't have to create a new logger code path. However, it creates a lot of noise and possibly memory overhead (Saving a large request body into some instrumentor class) and like @geemus mentioned it's a bit much for a one-off debug foray.
I don't have a strong opinion either way. If there was some way to say "Don't instrument this all the time" that would be good? Or turn it on and off somehow, but the way instrumentor is implemented, it doesn't seem that way. Or maybe I'm missing something.

@shaiguitar I think I want to step back for a second actually.  Can you describe in more detail a scenario that requires this (or at least makes you desire it).  I think understanding your use case better will make it easier to think about how best to provide for it.  Thanks!

Well say I have a third party library I'm working on that has Excon as a dependency. I'm making requests, and in my tests I want to see the raw request that is getting sent through. So I'm in a console and I want to be able to toggle something like
Excon.debug = true
See that debug output. Possibly I'd then be able to shut it (if I want to). Basically be able to see real requests being sent at runtime, in my case via some testing console.

@shaiguitar Needing the raw data seems to imply a lack of trust in building the actual requests though.  Which there may genuinely be (in which case we should address that).  If there is confidence in the translation from a request hash to a request though, this seems less needed (or at least not quite this same thing).  I at least find it quite rare that I would need to get at the raw side of things and find the already in a hash version much easier to deal with.

Yeah, I actually agree with what you're saying. It is lack of trust on my part, essentially because that's just the type of guy I am :) . Well, and:
Essentially while working on the new Terremark API I've run into some weird issue where the headers seem to be right, but they are denying 401 the request on the backend. So just to be extra cautious I wanted to see the raw request to see nothing is screwed up along the way.
For what it's worth, I started the work on https://github.com/shaiguitar/fog/blob/ecloudv2/lib/fog/ecloudv2/compute.rb - since they did a complete rewrite for the newer version(s), I'm starting off a different provider. We can then deprecate the older one once they deprecate it in their live API. I also want/am using Shindo instead of Rspec, granted that I don't understand why ecloud is the only one that is using rspec. You can compare both branches in case you are interested or if you think I may be going in the wrong way ( So, it's on: shaiguitar/fog/ecloudv2 ).
Last request I guess: to satisfy my lack of trusting, is there a way to develop fog with a local excon? What was your way of doing it?
I guess we can just close this then.

@shaiguitar Yeah, I've run into a few things like that.  What I do in that case is comment out the excon line in the gemspec and explicitly add it to the Gemfile instead (along with a path).  It is kind of a pain, but it doesn't (for me) come up very often.  I think that stuff is rspec because thats what ed did, I never felt a strong enough desire to mess with it to convert.  Not a good reason.  I'd sort of like to replace shindo with minitest, but that is a whole other can of worms.  If everything was at least in shindo for the time being that would be great.

Great, will do. Thanks

FWIW I've run into several scenarios where I needed to be able to view the raw data - I don't think that's an unreasonable use case at all. Typically what I do right now is proxy the requests through something like Charles proxy that can do inline decoding of SSL so I can see exactly what's coming through on both sides. It would be really nice for insecure requests to just be able to dump straight from Excon without the extra step of setting up a proxy.

@nextmat fair enough.  Thanks for the extra feedback.  I will continue to think about it.  It shouldn't be technically hard to implement, but it is likely to create some overhead somewhere.


FieldWithValue is not anywhere in the code

Package version: v9
Issue, Question or Enhancement:
FieldWithValue is in the godoc https://godoc.org/gopkg.in/go-playground/validator.v9, but does not exist anywhere in the source code.
Is the godoc out of date?

It was changed in v9 to VarWithValue to try and avoid some confusion as some people thought it had to be a struct field; must have missed updating it in the docs, thanks for reporting so I can correct them :)

@ryan-ju corrected in Release 9.9.4


Syntax error for passing kwargs to a function inside fstrings

Environment data

Language Server version: 2020.6.1
OS and version: Microsoft Windows [Version 10.0.19041.329]
Python version (& distribution if applicable, e.g. Anaconda): Python 3.7.4

Expected behaviour
Passing a dictionary to a function inside an fstring should not show any syntax errors.
Actual behaviour
Pylance shows syntax error when encountering a dict ({}) inside an fstring
Logs

Doubt the logs would be useful, but here they are anyway-
[Info  - 5:07:38 PM] Pylance language server 2020.6.1 starting
[Info  - 5:07:38 PM] Server root directory: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server
[Info  - 5:07:38 PM] No configuration file found.
[Info  - 5:07:38 PM] Setting pythonPath for service "test": "C:\Users\Chase\AppData\Local\Programs\Python\Python37-32\python.exe"
[Info  - 5:07:39 PM] stubPath c:\Users\Chase\Desktop\Python Projects\test\typings is not a valid directory.
[Info  - 5:07:39 PM] Assuming Python version 3.7
[Info  - 5:07:39 PM] Assuming Python platform Windows
[Info  - 5:07:39 PM] Searching for source files
[Info  - 5:07:39 PM] Found 1 source file
[Info  - 5:07:39 PM] Background analysis root directory: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server
[Info  - 5:07:39 PM] Background analysis started
[FG] parsing: c:\Users\Chase\Desktop\Python Projects\test\test.py (31ms)
[FG] parsing: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\2and3\builtins.pyi (377ms)
[FG] binding: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\2and3\builtins.pyi (156ms)
[FG] binding: c:\Users\Chase\Desktop\Python Projects\test\test.py (2ms)
[BG] analyzing: c:\Users\Chase\Desktop\Python Projects\test\test.py ...
[BG]   parsing: c:\Users\Chase\Desktop\Python Projects\test\test.py (169ms)
[BG]   parsing: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\2and3\builtins.pyi (380ms)
[BG]   binding: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\2and3\builtins.pyi (146ms)
[BG]   binding: c:\Users\Chase\Desktop\Python Projects\test\test.py (1ms)
[BG]   checking: c:\Users\Chase\Desktop\Python Projects\test\test.py ...
[BG]     parsing: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\3\typing.pyi (78ms)
[BG]     binding: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\3\typing.pyi (39ms)
[BG]     parsing: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\2and3\_typeshed\__init__.pyi (18ms)
[BG]     binding: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\2and3\_typeshed\__init__.pyi (3ms)
[BG]   checking: c:\Users\Chase\Desktop\Python Projects\test\test.py (187ms)
[BG] analyzing: c:\Users\Chase\Desktop\Python Projects\test\test.py (890ms)
[FG] parsing: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\3\typing.pyi (68ms)
[FG] binding: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\3\typing.pyi (36ms)
[FG] parsing: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\2and3\_typeshed\__init__.pyi (23ms)
[FG] binding: c:\Users\Chase\.vscode\extensions\ms-python.vscode-pylance-2020.6.1\server\typeshed-fallback\stdlib\2and3\_typeshed\__init__.pyi (8ms)

Code Snippet / Additional information

from typing import Dict

def foo(spam, dictval: Dict):
    print(dictval)
    return "Done"

print(f"{foo(0, {'bar' : 1, 'baz': 2})}")

Output after python ./test.py in the commandline-
{'bar': 1, 'baz': 2}
Done

The code doesn't actually cause a syntax error as you can see. But Pylance reports syntax error-

It also shows a parameter missing error, however that's certainly a side effect of the syntax error-

Thanks for the bug report. Pylance was not correctly handling nested braces within an f-string expression. It is now working. The fix will be in an upcoming release of Pylance.

This issue has been fixed in version 2020.7.0, which we've just released. You can find the changelog here: https://github.com/microsoft/pylance-release/blob/master/CHANGELOG.md#202070-9-july-2020


Incorporating feedback into getting_started.md

Summary
@carriewright11 and @KatherineCox gave feedback on the getting_started.md instructions on #41 and #29
This PR is to address those feedback points which are:
Intro

[x] Added resources for people unfamiliar with Git, markdown, and Rmarkdown including https://happygitwithr.com/

Set up branches section

[x] switch "click Add rule" instead of Add new rule
[x] need to say to check the Require status checks to pass before merging then say to check the Require branches to be up to date before merging underneath
[x] click create and save changes to finish out the set up branches section
[x] Tried to clear up the part about check_urls and style_n_check

Setting up the Docker image

[x] Added more explanation about what Docker is and why this is helpful
[x] Added link to @carriewright11 's explanation about Why Docker
[ ] Added @KatherineCox's wordings from this comment.

Citations section

[x] Added that people can use programs like Zotero/endnote
[x] Changed references.bib file  to book.bib because I meant to say that. lol.

Github Actions section

[x] Add why we don't require build-all

Add images section

[x] Add that images should be placed in resources/images/

There are two items from #41 brought up by @KatherineCox that are not addressed here yet, because I'm not sure about them:

What does the repo option in _bookdown.yml do, and what happens if I forget to change it? If this is important to set upfront, it'd be good to highlight it in the instructions.
Should LICENSE be updated to CC-BY, or should we have instructions that course developers should update the LICENSE (and leave the template under MIT)?

Any other comments to address before I merge this, @carriewright11?

It looks great! I wonder if we should add how to run a docker image on windows (docker run -it --rm -v %CD%:)? or a link about that

docker run -it --rm -v %CD%:

Good call! Will add!

Also I liked your sentence: "Particularly for courses that involve running example code, it's recommended that you use a Docker image for development to maintain software version consistency across course developers." I think it would be useful to add it back before where you link to my explanation. - good for people who might be somewhat familiar but needed an explanation as to why it would be useful in this setting.


Failed to start gauge API: Runner with pid 28497 quit unexpectedly(exit status 1)

gauge run mysql基础功能测试.md
crypto/x509
malformed DWARF TagVariable entry
FAIL    command-line-arguments [build failed]
FAIL
Failed to build project: Failed to compile project: exit status 2
Please ensure the project is in GOPATH.
Killing go runner.
Error ----------------------------------
[Gauge]
Failed to start gauge API: Runner with pid 28497 quit unexpectedly(exit status 1)
Get Support ----------------------------
Docs:          https://docs.gauge.org
Bugs:          https://github.com/getgauge/gauge/issues
Chat:          https://spectrum.chat/gauge
Your Environment Information -----------
darwin, 1.0.8, 28617ea
go (0.4.0), html-report (4.3.0), screenshot (0.1.0)
go version go1.13 darwin/amd64
gauge version
Gauge version: 1.0.8
Commit Hash: 28617ea
Plugins
go (0.4.0)
html-report (4.3.0)
screenshot (0.1.0)
my specs can not running,how to fix this Big?

Gauge 1.0.8 is over 4 years old.
Please upgrade your gauge version and plugins to latest versions and try again. If you still have issues, please comment and I'll re-open the issue.

Also, please note that Go 1.13 was EOL over 3.75 years ago. If you are working with specs implemented in golang as it looks like above, you should upgrade to a newer go version.


bottomTitleWidget or leftTitleWidget often  content is out of bounds or overlapping

The data dynamic span is large, and the minimum and maximum values cannot be controlled. I don't know how to handle this problem

Dup of #906


How to manager .ssh/known_hosts file.

When restarting the sftp service, SSH authentication fails. The warning logs:  WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
The detail log:
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ED25519 key sent by the remote host is
SHA256:4/usiLYoGeiUTd5NMxpS3Snm7gR5Vkudk4lnGR/5CS8.
Please contact your system administrator.
Add correct host key in /Users/baoyachi/.ssh/known_hosts to get rid of this message.
Offending ED25519 key in /Users/baoyachi/.ssh/known_hosts:32
Host key for [<IP_ADDRESS>]:2222 has changed and you have requested strict checking.
Host key verification failed.
Connection closed
Connection closed.

So, how to manage ~/.ssh/known_hosts file when sftp-server restart, that authentication fails.

Well, did you re-install the remote server, or are you trying to connect to a ssh server inside container?
In that case, the remote host identification does change and it will get rejected by ssh, I would suggest using a temporary ssh know_hosts file by using tempfile and SessionBuilder::user_known_hosts_file

Yeah, I used docker started sftp-server.

Yeah, I used docker started sftp-server.

I recommend you to use a tempfile as ssh known_hosta as suggested by my previous comment

Now, use /dev/null file replace.

Now, use /dev/null path replace.

Hmmm, does that work for you?

Yes.


Made Endpoint public so it can be used in custom routing.

Addresses issue #1514.

@tillig,
Thanks for having already signed the Contribution License Agreement. Your agreement has not been validated yet. We will now review your pull request.
Thanks,
.NET Foundation Pull Request Bot

Yep, this was just an oversight. Thanks


feature dimensions do not match

Hi, I'm trying to run the following command in prepare.sh,
CUDA_VISIBLE_DEVICES=6,7 ./tools/dist_train.sh  configs/lvis/prompt_save_train_reuse.py 2 --work-dir workdirs/prompt_save_train
and meet some errors like
Traceback (most recent call last):
  File "./tools/train.py", line 199, in <module>
    main()
  File "./tools/train.py", line 188, in main
    train_detector(
  File "/home/ubuntu/work/code/detpro/mmdet/apis/train.py", line 151, in train_detector
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
    self.run_iter(data_batch, train_mode=True)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter
    outputs = self.model.train_step(data_batch, self.optimizer,
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/parallel/distributed.py", line 46, in train_step
    output = self.module.train_step(*inputs[0], **kwargs[0])
  File "/home/ubuntu/work/code/detpro/mmdet/models/detectors/base.py", line 246, in train_step
    losses = self(**data)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
    return old_func(*args, **kwargs)
  File "/home/ubuntu/work/code/detpro/mmdet/models/detectors/base.py", line 180, in forward
    return self.forward_train(img,img_no_normalize, img_metas, **kwargs)
  File "/home/ubuntu/work/code/detpro/mmdet/models/detectors/mask_rcnn.py", line 83, in forward_train
    roi_losses = self.roi_head.forward_train(x, img, img_no_normalize, img_metas, proposal_list,proposals,
  File "/home/ubuntu/work/code/detpro/mmdet/models/roi_heads/standard_roi_head_collect_reuse.py", line 269, in forward_train
    bbox_results = self._bbox_forward_train(x,img,sampling_results,proposals_pre_computed,
  File "/home/ubuntu/work/code/detpro/mmdet/models/roi_heads/standard_roi_head_collect_reuse.py", line 364, in _bbox_forward_train
    bbox_results, region_embeddings = self._bbox_forward(x, rois)
  File "/home/ubuntu/work/code/detpro/mmdet/models/roi_heads/standard_roi_head_collect_reuse.py", line 305, in _bbox_forward
    bbox_pred = self.bbox_head(bbox_feats)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/work/code/detpro/mmdet/models/roi_heads/bbox_heads/convfc_bbox_head.py", line 214, in forward
    bbox_pred = self.fc_reg(x_reg) if self.with_reg else None
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 93, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/functional.py", line 1690, in linear
    ret = torch.addmm(bias, input, weight.t())
RuntimeError: mat1 dim 1 must match mat2 dim 0

It looks like the dimension of feature does not match the dimension of weight in the BBoxHead module. I guess because the shared layer is commented out in forward pass of the 'ConvFCBBoxHead' module. Can you help check the code?

Hi, I'm trying to run the following command in prepare.sh, CUDA_VISIBLE_DEVICES=6,7 ./tools/dist_train.sh configs/lvis/prompt_save_train_reuse.py 2 --work-dir workdirs/prompt_save_train and meet some errors like
Traceback (most recent call last):
  File "./tools/train.py", line 199, in <module>
    main()
  File "./tools/train.py", line 188, in main
    train_detector(
  File "/home/ubuntu/work/code/detpro/mmdet/apis/train.py", line 151, in train_detector
    runner.run(data_loaders, cfg.workflow, cfg.total_epochs)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 125, in run
    epoch_runner(data_loaders[i], **kwargs)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 50, in train
    self.run_iter(data_batch, train_mode=True)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/runner/epoch_based_runner.py", line 29, in run_iter
    outputs = self.model.train_step(data_batch, self.optimizer,
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/parallel/distributed.py", line 46, in train_step
    output = self.module.train_step(*inputs[0], **kwargs[0])
  File "/home/ubuntu/work/code/detpro/mmdet/models/detectors/base.py", line 246, in train_step
    losses = self(**data)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/mmcv/runner/fp16_utils.py", line 84, in new_func
    return old_func(*args, **kwargs)
  File "/home/ubuntu/work/code/detpro/mmdet/models/detectors/base.py", line 180, in forward
    return self.forward_train(img,img_no_normalize, img_metas, **kwargs)
  File "/home/ubuntu/work/code/detpro/mmdet/models/detectors/mask_rcnn.py", line 83, in forward_train
    roi_losses = self.roi_head.forward_train(x, img, img_no_normalize, img_metas, proposal_list,proposals,
  File "/home/ubuntu/work/code/detpro/mmdet/models/roi_heads/standard_roi_head_collect_reuse.py", line 269, in forward_train
    bbox_results = self._bbox_forward_train(x,img,sampling_results,proposals_pre_computed,
  File "/home/ubuntu/work/code/detpro/mmdet/models/roi_heads/standard_roi_head_collect_reuse.py", line 364, in _bbox_forward_train
    bbox_results, region_embeddings = self._bbox_forward(x, rois)
  File "/home/ubuntu/work/code/detpro/mmdet/models/roi_heads/standard_roi_head_collect_reuse.py", line 305, in _bbox_forward
    bbox_pred = self.bbox_head(bbox_feats)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/work/code/detpro/mmdet/models/roi_heads/bbox_heads/convfc_bbox_head.py", line 214, in forward
    bbox_pred = self.fc_reg(x_reg) if self.with_reg else None
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/modules/module.py", line 727, in _call_impl
    result = self.forward(*input, **kwargs)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/modules/linear.py", line 93, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/ubuntu/anaconda3/envs/detpro/lib/python3.8/site-packages/torch/nn/functional.py", line 1690, in linear
    ret = torch.addmm(bias, input, weight.t())
RuntimeError: mat1 dim 1 must match mat2 dim 0

It looks like the dimension of feature does not match the dimension of weight in the BBoxHead module. I guess because the shared layer is commented out in forward pass of the 'ConvFCBBoxHead' module. Can you help check the code?

I have fixed, please check.

I have fixed, please check.

Ok, I'll check it. BTW, I already run the prepare.sh and generated the 'lvis_clip_image_proposal_embedding' folder as below. Is this right? And will this folder be used if I just want to reproduce the results of ViLD?
lvis_clip_image_proposal_embedding
├── train
│   └── train2017
└── val
    ├── train2017
    └── val2017

I have fixed, please check.

Ok, I'll check it. BTW, I already run the prepare.sh and generated the 'lvis_clip_image_proposal_embedding' folder as below. Is this right? And will this folder be used if I just want to reproduce the results of ViLD?
lvis_clip_image_proposal_embedding
├── train
│   └── train2017
└── val
    ├── train2017
    └── val2017

It is correct. 2) The zip file of this folder will be used.


Throw to a svelte component from hooks

Describe the problem
My use case is error management for protected routes. As discussed in #6315, the best practice is to place the protection logic in hooks.server.ts. Let's say I want to distinguish between different protection levels (verified, premium, admin etc). There are currently two options:

throw error(403, {code: 'verified'}) from hooks.server
Then the error.html file is served and it's hard to customize, there is no tooling available:

I will struggle to use my Tailwind classes by finding a way to point to the processed css files
I will have to use raw js to manipulated the DOM to distinguish between my hard-coded cases (verified etc.) and show relevant divs and links.

throw redirect(303, "/protected?code=verified") from hooks.server
This way I can use a svelte component, styling is easy, and I can access my error codes from app.d.ts

But isn't is weird to return 303 instead of 403 ? (and since it's a server redirect it will just end up being a 200 to the /protected route)
Also I have to manually write the error code in the redirect url which doesn't feel very idiomatic

Is there a better way?
Describe the proposed solution
Maybe a +protected.svelte component (or with any other name...) on the root level where a call to error inside hooks would lead? Or maybe make error lead to the root +error.svelte?
Which would basically allow to use throw error(403, {code: "verified"}) from hooks in a more manageable way and would return the correct 403 error status
Alternatives considered
Option 2 seems better for now
Importance
would make my life easier
Additional Information
No response

I'd go with 2nd solution: the status code doesn't matter much if you respond with a HTML page - the user anyway won't see the status code.

A redirect is the correct solution here - it's what you're doing, you don't allow people to visit the original page, you want them to log in somewhere else.


Use FIFO queues to batch scheduled operations across tables

Background
We previously converted a lot of lambdas that run on CloudWatch rules to spread out the invocation across tables by creating batches of tables and putting them on an SQS queue:

https://github.com/gchq/sleeper/issues/1607

This runs the risk of overlapping invocations of those lambdas on the same tables.
Description
We'd like to use FIFO queues to ensure that only one instance of a lambda is running a scheduled action at a time for each Sleeper table.
Analysis
https://aws.amazon.com/blogs/compute/new-for-aws-lambda-sqs-fifo-as-an-event-source/
https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-fifo-queues.html
For scheduled actions, we probably want to avoid multiple invocations queuing up. We can think about how that should work.
One thing we could do is set a batch size on the trigger to consume the FIFO queue. That would allow multiple tables to be assigned to the same lambda up to the batch size, but it would also mean that if there are multiple messages for the same table, they would be taken off the queue at the same time and the lambda could ignore the duplicates.
We can default the batch size to 1 and make it configurable. We could make a separate issue to review that.
We can split out separate issues for each operation that needs converting:

https://github.com/gchq/sleeper/issues/2352
https://github.com/gchq/sleeper/issues/2370
https://github.com/gchq/sleeper/issues/2371
https://github.com/gchq/sleeper/issues/2372

Merged accidentally, it was a draft PR on a commit that was included in the PR for compaction job creation.

Closing as all the issues split from this are finished.


Add Troubleshooting Section and Fix Minor Typos

I’d like to propose adding a troubleshooting section to the README to help users resolve common issues they might encounter while setting up or using the Predictive Calc application. This could include FAQs or solutions for typical errors.
Additionally, I noticed a minor typo in the README: "Futher" should be corrected to "Further."
If possible, could you please assign this issue to me? I’d be happy to implement these changes!

This issue isn't needed at the moment. Please review the existing issues or open a new one, relevant to the project and AI/ML if necessary.


Integrate twilio

Integrate twilio as a notifier
Write test cases for twilio service
Update example yml files

closes #7

Codecov Report

Merging #18 into master will decrease coverage by 9.74%.
The diff coverage is 41.81%.

@@            Coverage Diff             @@
##           master      #18      +/-   ##
==========================================
- Coverage   84.67%   74.92%   -9.75%
==========================================
  Files           8       10       +2
  Lines         261      347      +86
  Branches       37       50      +13
==========================================
+ Hits          221      260      +39
- Misses         40       87      +47

Impacted Files
Coverage Δ

src/config/default.config.js
100% <ø> (ø)
:arrow_up:

src/utils/twilioClient.js
100% <100%> (ø)

src/services/notifier.js
89.47% <100%> (+0.58%)
:arrow_up:

src/services/twilio.js
32.85% <30.43%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fddf294...de7732c. Read the comment docs.

Looks all good to me :+1:


Some error in predict~~

We are getting error in predicting single images after training these models, the predicted images are black, can you please upload the script for predicting single images as well, thanks a lot!

mask = output.cpu().detach().numpy()[0][0] * 255


Broken Link

This link this broken: http://carbondesignsystem.com/style/iconography/library

Thanks for the heads up! 🙏


Update introduction.md

🚀 Deployed on https://639a5fdfba39bc651e14bd69--cozy-fox-0defec.netlify.app


"Copy as Image" menu doesn't show up in the context menu in the Firefox browser

Preflight Checklist

[x] I agree to follow the Code of Conduct that this project adheres to.
[x] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

Describe the bug
A clear and concise description of what the bug is.
I can't copy the selected components as an image in the Firefox browser
I want to copy&paste selected components as an image from the draw.io to out of the draw.io (e.g. Slack, Web browser, Notion, etc)
It can be done by the "Copy as Image" menu in the Chrome browser, but the menu doesn't show up in the Firefox browser.
I already have tried Cmd-C to copy as an iamge, but I just got the formatted text in the Clipboard, instead of the image.
To Reproduce
Steps to reproduce the behavior:

Open a new diagram (in the Firefox browser)
Draw some components e.g. jsut a rect.
Select the rect and right-click, you can see the context menu.
"Copy as Image" doesn't show up in the context menu.

Expected behavior
A clear and concise description of what you expected to happen.
"Copy as Image" should show up in the context menu like doing it in the Chrome.
Screenshots
If applicable, add screenshots to help explain your problem.

in Chrome

in Firefox

draw.io version (In the Help->About menu of the draw.io editor):

20.5.3

Desktop (please complete the following information):

OS: [e.g. iOS] macOS
Browser [e.g. chrome, safari]: Firefox
Version [e.g. 22]: 107.0

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

Unfortunately, Firefox does not support this function.


Changed import of @yamada-ui/lucide in forms of Storybook to *Icon.

Description
Correct the writing style deprecated with this PR.
#3358
For example, fix the parts in stories/components/data-display/calendar.stories.tsx that are not imported in *Icon format.
- import { Ghost } from "@yamada-ui/lucide"
+ import { GhostIcon } from "@yamada-ui/lucide"

Problem Statement/Justification
Because it's deprecated.
Proposed Solution or API
No response
Alternatives
No response
Are you willing to participate in implementing this feature and create a pull request include the implementation?
None
Additional Information
No response

Can I work on this?

Can I work on this?

Yes Please! @Ponraaj

@illionillion  I have raised PR for this issue. Kindly check it out


Erroneous translation DE->en-GB and en-US

Describe the bug
A German contract document (PDF) contains the words AUFTRAGGEBER (Client) and AUFTRAGNEHMER (provider). These 2 German words are both translated to CONTRACTOR or both to CUSTOMER, effectively corrupting the resulting document.
Adding the correct translation to a glossary does not change the behaviour.
Testing a empty Word document with only these 2 words and a glossary does translate correctly.
To Reproduce
Steps to reproduce the behavior:
Have a PDF document with these 2 words in amongst many other text
Post it to the API with or without a glossary
The result is as described above
Expected behavior
Both words should not be translated the same AND the glossary should enforce the right translation.
Screenshots or sample code
Original
Translated
** Additional context **
Selecting the one sentence and translating with the desktop app gives the exact same bad result.

Hi @boeryepes69, thank you for reporting the issue.
I could reproduce it, and you are right: this behavior is unexpected. I'll forward it to the relevant team and ask if there is a solution.

Thanks for the report! In case you can change the document, avoiding the upper casing fixes this for me:
"Jegliche Verlagerung von Betriebsaktivitäten des Auftragnehmers im Bezug auf Services erfolgt anhand eines vom Auftragnehmer erstellten und Auftraggeber genehmigten, formellen Migrationsplans für die ordnungsgemäße TRANSITION und SERVICE-MIGRATION sowie ununterbrochenen Service."
gets translated to
"Any relocation of the Contractor's operations with respect to Services shall be based on a formal migration plan for proper TRANSITION and SERVICE MIGRATION and uninterrupted service prepared by the Contractor and approved by the Client."
We'll check why glossaries don't work for this and if we can fix the translation problem itself.

i confirm JanEbbing's observation that without capitals, the translation works. It's really weird that capitals make such a difference.
The glossaries not working is interesting and I believe issue #16 is related (it's the same me but I lost access to the other email when I switched providers ...).
I can also confirm that when I pass the text to the DeepL API, the correct GlossaryId is included in the TextTranslateOptions object.


Replace npm lint with eslint

The current solution for linting via npm run lint isn't secure, as it allows arbitrary code execution. This replaces it with a global eslint installation. At the moment, eslint is used for all node applications. In the future, we could provide support for other JS linters via some sort of config.

 Replace JS linter with pre-installed eslint, jshint, etc


Manage running goroutines and graceful shutdown

Motivation
Graceful shutdown is proving to be a challenge when multiple goroutines are running in the background. This has two parts:

Signaling a shutdown and having each service respond by terminating gracefully (this part is working, but each service has its own, slightly different, implementation).
being able to know when all services have completed.

This is important in production, where we want to ensure no data is lost due to a dirty shutdown, and in tests where we want to terminate quickly and cleanly without having to add unnecessary waiting periods.
Method
We want to integrate Tomb, a package for handling clean goroutine tracking and termination.
Tomb provides a Go() method that's intended to replace calling the go keyword directly to start goroutines. Internally, it uses a waitgroup to track how many goroutines have been started and how many have completed. It can also provide the first error that triggered a shutdown, if it was due to an error.
Tomb should be integrated in a single module first and then additional PRs can integrate it into more modules. Eventually:

No goroutine should be started using a bare go keyword.
All method calls that can accept a Context should receive one, provided by Tomb.
All select statements should have an early termination clause using tomb.Dying() (a method returning a channel that's closed when the Tomb is killed). If this clause is invoked, an ErrDying (a Tomb constant) should be returned by the goroutine and Tomb knows to ignore it as a kill reason.
All received context objects (specifically in api handlers) should be wrapped using tomb.Context(ctx) if used (I think they're never used as of writing this).
New contexts should never be generated from scratch and we should always use the Tomb for context (this happens mostly in the P2P module, but also in the PoET client).

WIP

[ ] Add specific tasks (where to initialize the Tomb, how to do shutdown)
[ ] List some candidate modules for the first integration

I'd love to work on this task, as it would be a good opportunity to better familiarize with a high-level code and services' organization.

No goroutine should be started using a bare go keyword.
This could be checked/enforced by a simple linter that analyses code and looks for bare go keywords, perhaps? There are clearly will be exemptions, but if the rules are clear, this can be neatly codified.

Hi @divan, sounds good. Feel free to give this a shot. It would probably be best to propose a plan before beginning implementation work - I'm happy to review it. We'll try to fill in some more details in this issue as well.

Related: #514

Related: https://github.com/spacemeshos/go-spacemesh/issues/1639

@noamnelke still relevant?

This isn't a task for Noam, I think @dshulyak @countvonzero should weigh in here from a node architecture perspective

i still agree with the general approach described in the issue. there is just one caveat.
currently in the node we are using golang.org/x/sync/errgroup to manage goroutine.
the aforementioned Tomb was attempted and discarded in favor of errgroup.
what remains to be done:

update all modules that spawn goroutines to use errgroup
use context to manage app shutdown
audit the code for panic stmts and see whether they are compatible with graceful shutdown


Illicit Drug Grows

id=a4c628fc4aa04a05ad30fd32613df340

[x] If this is a demo item only, change the sharing to Demo Review instead of Release Review

or

[ ] If this is a release item, elaborate in the description to be more specific (where, how, how is the information presented, etc.)

id=f6e165bf688c46daa06048fed1fb43e9

[x] The description makes mention of filtering and summarizing, but I don't see that functionality when I View the Application.

Fixed it.

Count and Summary work fine, but now it doesn't summarize by plant (that I can see). Pop-ups now just list count and sum.

This item has been released already. I am closing this issue.


Add Kwik EFIS / DMAP to apps with stratux support

Kwik EFIS for Android has been upgraded to support Stratux.
The application is released as open source and made available through F-Droid as a signed and fully FOSS compliant package.
Website: http://members.iinet.net.au/~ninelima/efis
Repository: https://github.com/ninelima/kwikEFIS

Already done.


Inconsistent organization member lists, return Person instead of OrganizationMember

Needed by https://github.com/opengisch/qfieldcloud-private/pull/460.

Task linked: QF-2365 Member list different depending where you are

Thanks!


SEMM update

@dansimp A community contributed PR is available for review. Thanks.

@Dansimp Would you be able to review this community contributed PR as the time permits? Thank you.


Add graph structure generation

We must be able to take a list of images defined with cognitive atlas tags, and output a data structure that can be used for reverse inference analysis and visualization of result.

Closing - this should be done in the Cognitive Atlas API, and pybraincompare handles the similarity calculation.


Enhancement: SilverStripe 4 compatibility

This is a placeholder issue for SS4 compatibility.
Pre-requisites

[ ] Branch current master off to 1 or similar for maintaining the current version
[ ] Update/add composer branch alias to 2.0.x-dev

Requirements

[ ] Add namespacing for module including upgrader mapping file
[x] PSR-2 code formatting (looks like its mostly done)
[ ] Implement core namespaces, update configuration (including CI builds), tests and template locations
[ ] Add PSR-4 autoloader (inc. move code/ to src/ for consistency)
[ ] Update documentation

Implications

Minimum PHP version becomes 5.5

@dljoseph if you wouldn't mind doing the branching and update the composer alias, I'm happy to make PRs for the rest of the changes required.
"extra": {
    "branch-alias": {
        "dev-master": "2.0.x-dev"
    }
}

Hi Robbie, Thanks for this.  Sorry I'm a bit late in responding.  I'll do that today.

Hi @robbieaverill that's done.  Branch 2.0 created, composer.json branch-alias added.  Go ahead and submit your PRs.


chore(result): Adding data field inside chaosresult

Signed-off-by: shubhamchaudhary<EMAIL_ADDRESS>
What this PR does / why we need it:
Which issue this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #
Special notes for your reviewer:
Checklist:

[ ] Fixes #
[ ] Labelled this PR & related issue with documentation tag
[ ] PR messages has document related information
[ ] Labelled this PR & related issue with breaking-changes tag
[ ] PR messages has breaking changes related information
[ ] Labelled this PR & related issue with requires-upgrade tag
[ ] PR messages has upgrade related information
[ ] Commit has unit tests
[ ] Commit has integration tests

This PR will be closed in favour of a changed approach toward exposing context/specific experiment run data. Discussions will be tracked/updated here: https://github.com/litmuschaos/litmus/issues/1254


MuTect1

@MaxUlysse and @Sebastian-D pls have a look - it is a duplication of MuTect2 practically. On long term we have to refactor these into one, especially the collate part.

Looks good, I see the influences ;).
I am doing something similar with HaplotypeCaller, the collateFiles function is bothering me however as I need to change it for normals and I am not proficient at groovy, yet.


fix #17914 ActiveRelationTrait compatibility with php 7.4

Fix for PHP 7.4
Trying to access array offset on value of type null
To support "Array-style access of non-arrays" in php 7.4: https://www.php.net/manual/en/migration74.incompatible.php
See previous conversation here: #17769

Q
A

Is bugfix?
✔

New feature?
❌

Breaks BC?
❌

Tests pass?
✔

Fixed issues
#17914 #17769

Changelog was updated

Thank you!


Feat: added refreshAndAwait() to set refresh and await until the loading completes

Added .refreshAndAwait() to FutureSignal in order to do .refresh() but instead of it just changing the state it awaits the future to recompute and finishes the refreshing
good for drag down to refresh indicators and other loading indicators that rely on completing futures
since isRefreshing/isReloading is different than isLoading

Why would you not just await a .refresh/reload?

.refresh/reload does wait for the internal execution to complete. It just updates the state on each update

.refresh/reload does wait for the internal execution to complete. It just updates the state on each update

It doesn't I tried it.
even the code it doesn't await any asynchronous calls.

That should be unintended behavior then. I will write a test to validate expectations first

any updates ?

Preparing for IO connect but should be able to address in the next few weeks

Hello @rodydavis I've came around again after your recent releases and this issue still persists.
awaiting .refresh or .reload does not waits for the future/computation to finish, i have to rely on my implementation to actually await for the future to recompute again
can we reopen this PR or do something about it?


Added quick queue support for 8muses.download

Category
This change is exactly one of the following (please change [ ] to [x]) to indicate which:

[X] a refactoring

Description
I added quick queue support for 8muses.download search pages and category pages
Testing
Required verification:

[ ] I've verified that there are no regressions in mvn test (there are no new failures or errors).
[ ] I've verified that this change works as intended.

[ ] Downloads all relevant content.
[ ] Downloads content from multiple pages (as necessary or appropriate).
[ ] Saves content at reasonable file names (e.g. page titles or content IDs) to help easily browse downloaded content.

[ ] I've verified that this change did not break existing functionality (especially in the Ripper I modified).

Optional but recommended:

[ ] I've added a unit test to cover my change.

Coverage decreased (-0.003%) to 35.869% when pulling e541d1cba0e20ece318e45e8ee54e0c075c1db73 on cyian-1756:Eightmuses.downloadQuickQ into<PHONE_NUMBER>6d440c3ad1697ad340890f4f157dd0 on RipMeApp:master.


Error messages sometimes fail to disappear

At first they disappear after a short while, but sometimes they get stuck

I think we close this one as it doesn't seem to happen any more


temporal-system:read role should have access to ListSearchAttributes RO API (when using default authorizer)

Expected Behavior
When using Temporal's default JWT authorizer, users with temporal-system:read permission should have access to temporal.api.operatorservice.v1.OperatorService/ListSearchAttributes API
https://github.com/temporalio/temporal/blob/release/v1.21.x/common/authorization/frontend_api.go#L50
Actual Behavior
Users with temporal-system:read permission do NOT have access to the ListSearchAttributes API. This can be confirmed via CLI (temporal operator search-attribute list) or just by opening Temporal UI.
In case of CLI we get Request unauthorized error, when opening Temporal UI we get redirected back to the login screen after authentication because of the 403 Forbidden error to the /search-attributes endpoint
Steps to Reproduce the Problem

Deploy Temporal with default authorizer
Configure an Oauth2 client in your auth provider to add permissions: [ "temporal-system:read" ] claim to the access token.
Request to list search attributes via tctl or temporal cli with your access token or open Temporal UI.

Specifications

Version: Temporal v1.21.4

Thanks for reporting this issue! It appears that the temporal-system:read permission is insufficient to access the ListSearchAttributes API, even though the documentation suggests otherwise.
This discrepancy might be a bug in Temporal's authorization logic, and we're actively investigating it.
In the meantime, here's a potential workaround:

Grant the user the temporal-system:manage permission instead of temporal-system:read. This permission provides broader access and should allow the user to list search attributes.

Please note that this workaround might grant the user more permissions than intended, so carefully consider its implications before implementing it.
We'll keep you updated on any progress made towards resolving this issue.

Thanks for reporting this issue!
It looks like you're experiencing an issue with the default JWT authorizer in Temporal not granting access to the ListSearchAttributes API for users with temporal-system:read permissions.
To confirm, are you using a custom JWT authorizer or the default Temporal authorizer?
If you're using the default authorizer and the issue persists, could you please provide the following information:

The exact version of the Temporal server you're using.
The configuration settings for your JWT authorizer (specifically, the permissionsClaimName setting).
A code snippet showing how you're starting your Temporal server and configuring the JWT authorizer.

Please let me know so I can further assist you in finding a solution.

Thank you for reporting this issue.
This seems to be a known issue that we're working on fixing. We have a documentation page with some workarounds that you can use for now. You can find more information about this issue and how to resolve it here.


SocketCAN broken since #7616

Since the changes from PR #7616 it seems that SocketCAN support is borken.
When transmitting a can frame using cansend it will keep sending the message indefinitely, it seems that priv->dev.d_len stays != 0.
Reading a frame using candump returns read: incomplete CAN frame  it seems that recvmsg() returns 0 bytes.

@anchao could you take a look?

I've found the workaround to the transmit problem it seems with new IOB api we've to explicitely set d_len to 0. Not sure about the other drivers that are affected by this.
diff --git a/arch/arm/src/s32k3xx/s32k3xx_flexcan.c b/arch/arm/src/s32k3xx/s32k3xx_flexcan.c
index 3e17b52ee0..d0ac32894d 100644
--- a/arch/arm/src/s32k3xx/s32k3xx_flexcan.c
+++ b/arch/arm/src/s32k3xx/s32k3xx_flexcan.c
@@ -905,6 +905,8 @@ static int s32k3xx_transmit(struct s32k3xx_driver_s *priv)

   mb->cs = cs; /* Go. */

+  priv->dev.d_len = 0;
+
   regval = getreg32(priv->base + S32K3XX_CAN_IMASK1_OFFSET);
   regval |= mb_bit;
   putreg32(regval, priv->base + S32K3XX_CAN_IMASK1_OFFSET);

I've found the workaround to the transmit problem it seems with new IOB api we've to explicitely set d_len to 0. Not sure about the other drivers that are affected by this.

It seems to be the same issue as https://github.com/apache/nuttx/issues/7802 / https://github.com/apache/nuttx/pull/7807, I think d_len should be set to 0 after device callback,  I will submit a patch to resolve this issue from the common code

@PetervdPerk-NXP  I resubmit the https://github.com/apache/nuttx/pull/7807, please try this change if it can resolves your issue

@PetervdPerk-NXP could you please confirm it is working now and please close this issue

@acassis @PetervdPerk-NXP reply in #7807. It work, so let's close it.


Pin CocoaLumberjackSwift to Swift 2.3 for other targets too

New Pull Request Checklist

[x] I have read and understood the CONTRIBUTING guide

[x] I have read the Documentation

[x] I have searched for a similar pull request in the project and found none

[x] I have updated this branch with the latest master to avoid conflicts (via merge from master or rebase)

n/a I have added the required tests to prove the fix/feature I am adding

n/a I have updated the documentation (if necesarry)

[x] I have run the tests and they pass

[x] I have run the lint and it passes (pod lib lint) - caveat: only on CocoaPods 1.1.0rc2, I think because of this https://github.com/CocoaPods/CocoaPods/pull/5540

Pull Request Description
Specify use of Swift 2.3, fixing compilation for iOS in Xcode 8. I ran into this when trying to use CocoaLumberjack via Carthage.
Alternately, the setting could be specified on the project file itself. I saw an earlier commit that set it per-target and followed their lead.

This PR will fix issues: #772 #766

It will not fix #772 as that one suggests to move to swift 3.0

Swift/3.0 suggested in comments, but Swift/3.0 still in WIP state.
This PR is ready for merge and will fix compile issues with Xcode8.

Is there an estimate for when this will get merged? Thanks for all the work 👍


Line Height when given a unitless value doesn't return as expected

G'day, love BSS - keep up the good work 🙌
I'm trying to use a unitless value for line-height (MDN) however bss is adding 'px'.
For example, when I pass in 1.25 as an int or the string '1.25', I am getting 1.25px however I want 1.25 (uses the number multiplied by the element's font size).

Thx a lot Ryan 😊
I've got an issue mentioning that, I'll just try to find, and there's a fix in the upcoming v2, but until then you can get around it by wrapping your value in calc

Thanks a lot! Could also use 125% as well, I just thought I'd mention it 😎

Yeah, I think we should just keep this issue open, I couldn't find one specifically for line-height :)
btw, percent might not mean exactly what you think ;) https://developer.mozilla.org/en-US/docs/Web/CSS/line-height#Prefer_unitless_numbers_for_line-height_values


Support iSCSI utility installation (targetcli) v2

This series adds support for iSCSI component installation, in preparation for clustered RBD/LIO iSCSI target deployment:
https://wiki.ceph.com/Planning/Blueprints/Hammer/Clustered_SCSI_target_using_RBD
Changes since v1 (https://github.com/ceph/ceph-deploy/pull/309)

Added test_install_only_iscsi unit test
Rebased against master

Refer to this link for build results (access rights to CI server needed):
http://jenkins.ceph.com//job/ceph-deploy-pull-requests/1700/
Test PASSed.

@ddiss This seems to just use --iscsi as a shortcut to installing targetcli.
I think the main question is whether abstracting the ultimate package name away from the user is beneficial.  I'm hesitant to pull this in because targetcli is not a Ceph package, and ceph-deploy already has generic commands for installing packages by name.
You could do:
ceph-deploy pkg --install targetcli <host> for example.

I'd hoped to piggy-back on the "install --iscsi" behaviour to also perform sanity checks for the corresponding Ceph RBD and LIO kernel modules. However, this could be done outside of ceph-deploy, and I agree that the "ceph-deploy pkg --install" functionality should be sufficient.
Thanks for the feedback Travis.


Update OurUsers.res for adding an logo of www.mnggiflab.com

logo file is missing

@Mng12345 Seems like you need to reformat the edited ReScript files for the tests to pass.

Also, is this your personal website, or is this an actual company? We'd rather only put businesses in this section, since it's about commercial proof.


Update github urls

@davisagli we may need to do some work to get readthedocs updated again?

@cdcarter the readthedocs link seems to be still working fine.

nice!


Warning: "Cannot read property '0' of null" when jasmine_webpack task is executed

Hi folks,
I´m using grunt-jasmine-webpack version 1.0.1 on Windows and it always fails with the following warning:
Running "jasmine_webpack:test" (jasmine_webpack) task
Warning: Cannot read property '0' of null Use --force to continue.

I found out that the problem is on line 27 in jasmine_webpack.js, where the following statement is executed:
var promisePolyfillPath = require.resolve('es6-promise-polyfill').match(/.*\/node_modules\/[^/]+\//)[0] + "promise.min.js"
This fails on my system since the path contains backslashes instead of forward slashes (due to windows?).
I´m using the following node and grunt versions:
grunt-cli v1.2.0
grunt v0.4.5
node v6.10.2
Can you please have a look at this issue?
Thank you & best regards,
Christof.

@cluick Thanks for reporting - does that PR (#25) fix the issue for you? I don't have grunt setup on any of my Windows VMs at the moment.

The issue is resolved with your change. Thank you very much for fixing this immediately!

1.0.2 is now published with this fix


Feed/Events - show list of groups/pages that have been crawled

Show the input of groups/pages that the crawler uses for the feed and events.
As of now, the feed is populated with data coming only from the PAC group. But there could be more groups taken into account later.
Same for pages and the crawled events.

List of sources for the feed is shown.
Issue addressed by @vigliag

Done for /feed


Fullscreen mode not working in image viewer for essay and splash pages.

Before proceeding, make sure there isn’t an existing issue for this bug.

[X] I have searched the existing issues and determined this is a new bug.

Expected Behavior
No matter the layout type, when an image is opened in the image viewer, and you click the fullscreen button in the upper left corner, it should open the image in fullscreen mode.
Actual Behavior
On the essay and splash pages, when the image is opened in the viewer and the fullscreen button is pushed, nothing happens. The image does not go fullscreen.
Important to note: you can open images in fullscreen mode on the entry type pages.
Steps to Reproduce

Start a new quire project
Navigate to any of the images on the Introduction  (splash) or American Photographs (essay) pages, open the images in the image viewer and click the fullscreen button

Version Numbers
[test-project]
quire-cli 1.0.0-rc.10
quire-11ty 1.0.0-rc.14
starter<EMAIL_ADDRESS>[System]
quire-cli 1.0.0-rc.10
node v18.16.0
npm 9.5.1
os Darwin 21.6.0
Web Browser
I tested with Chrome 114 and Firefox 117, and @cbutcosk tested on Safari 16.6 and Chrome 116.
Relevant Terminal/Shell Output
No response
Supporting Information
Special thank you to @cbutcosk for bringing this bug to our attention in #815.  As noted by @cbutcosk "I suspect your developers also need to add a Shadow DOM polyfill for Safari < 16.5 (mobile and desktop), which in this issue breaks in a slightly different way from the Chromium-based browsers."
So when fixing this issue be sure to test in multiple browsers!

I had a chance to peer into this, it turns out this line is the culprit on browsers that should otherwise support the fullscreen API: https://github.com/thegetty/quire/blob/e36d4cb100a81fd6ecd90a894e78f14e47e1b11b/packages/11ty/_includes/web-components/lightbox/index.js#L80
Optional-chaining the wrapper and classList here lets the function succeed and open fullscreen as you'd expect. I didn't have a shadowDOM-unsupporting browser available but fullscreen did still work when I disabled shadowDOM on Safari 17.0 for whatever that is worth.


[flash_ctrl/dv] Randomize read, program and erase latencies

see this comment
It would be very beneficial to randomize the read / program / erase latencies in the flash block level tests.
This will help ensure the design is robust against a different range of timing.

Enabled by #14745


submission pipeline - data not being saved

submit a project
Add keywords after the submission
click "Manage studies"
You see the submission pipeline
The added keywords are not there

Update the draft keyword based on the project keywords.

Working as expected.


fix: fix output for yaml and json, banner

fix ouput for yaml and json with verbose
do not show banner on yaml or json

Codecov Report
Attention: 7 lines in your changes are missing coverage. Please review.

Comparison is base (2299588) 48.06% compared to head (37e12d3) 48.16%.
Report is 1 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #149      +/-   ##
==========================================
+ Coverage   48.06%   48.16%   +0.10%
==========================================
  Files          20       20
  Lines        1783     1802      +19
==========================================
+ Hits          857      868      +11
- Misses        785      789       +4
- Partials      141      145       +4

Files
Coverage Δ

pkg/kor/confimgmaps.go
67.21% <100.00%> (+0.27%)
:arrow_up:

pkg/kor/deployments.go
56.00% <100.00%> (+0.89%)
:arrow_up:

pkg/kor/hpas.go
50.56% <100.00%> (+0.56%)
:arrow_up:

pkg/kor/ingresses.go
61.05% <100.00%> (+0.41%)
:arrow_up:

pkg/kor/pdbs.go
52.30% <100.00%> (+0.74%)
:arrow_up:

pkg/kor/pvc.go
56.06% <100.00%> (+0.67%)
:arrow_up:

pkg/kor/roles.go
59.21% <100.00%> (+0.54%)
:arrow_up:

pkg/kor/secrets.go
70.16% <100.00%> (+0.24%)
:arrow_up:

pkg/kor/statefulsets.go
60.00% <100.00%> (+0.81%)
:arrow_up:

pkg/kor/serviceaccounts.go
62.40% <33.33%> (-1.77%)
:arrow_down:

... and 2 more

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Per-row scores should increase with block fall rate

Decided not to do this, but changed to increase speed after every 10 completed lines, rather than after time intervals.


setVariables causes getCurrentResult to return wrong result set

observableQuery.variables != observableQuery.options.variables after setVariables is called. setVariables only sets observableQuery.variables, but observableQuery.getCurrentResult() uses observableQuery.options.variables, thus returning an incorrect result (the result for the query with the first set of variables).

Wherever possible we should try to make sure that there's only one code path that accomplishes a certain thing. So setVariables for example should rely on setOptions and contain practically no code. I'm not even sure if we should keep setVariables around at all.
Error handling is similar in some ways because there are multiple places in which errors get recorded and passed to the UI. This makes things a little bit brittle, because every time someone modifies the code, they must remember to do it in multiple places in a consistent way.


chore: prevent normalization of semver versioning

When there is a patch version added to semver versioning, setuptools.setup(version) will normalize the versioning from -patch to .patch which is not correct SEMVER versioning. The added feature with setuptools.sic(version) will prevent this from happening.

closing this as we don't need to strictly follow semver guidelines


move buildspecs to main

Issue #, if available:
https://github.com/aws/eks-distro-internal/issues/269
Description of changes:
final step in our conformance-test-in-codebuild switchover -- move the target branch in the codebuild to main.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/approve


ETL freezing on step data://meadow/climate/2023-12-20/surface_temperature

ETL tends to freeze on step data://meadow/climate/2023-12-20/surface_temperature. The snapshot has 1.6gb, so ETL may start swapping memory after decompressing it.
I can't run it locally as the memory spikes and starts swapping. It takes ~3min on @veronikasamborska1994 machine.

@veronikasamborska1994 I looked at the script and couldn't find any way how to make it faster. I tried to optimise memory usage a bit in this PR, but the peak memory usage could still be >20gb (I've seen 40gb of memory on some staging servers). The memory spikes after creating clip object.
It takes ~2h to run the step on the server. I don't know why it is so long. It could be memory swapping, but it could be as well something else.
Is such a heavy computation expected? Is there any way to make it faster?

@Marigold I'll take a look but there aren't that many ways in which this clipping can be done so I am not sure if I'll be able to improve it much.
What I could definitely do though is move this processing step to snapshot? Would that help?

What I could definitely do though is move this processing step to snapshot? Would that help?

That's a great idea! It's definitely easier than figuring out why it is so slow in production.

Also getting this from CI run ETL - build public datasets (branch) (I didn't know there's a memory cap for CI or Buildkite agents?)
File "/var/lib/buildkite-agent/builds/etl-build-2-15/our-world-in-data/etl-build-public-datasets-branch/etl/steps/data/meadow/climate/2023-12-20/surface_temperature.py", line 77, in run
  clip = da.rio.clip(data.geometry.apply(mapping), data.crs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/buildkite-agent/builds/etl-build-2-15/our-world-in-data/etl-build-public-datasets-branch/.venv/lib/python3.11/site-packages/rioxarray/raster_array.py", line 911, in clip
  cropped_ds = _clip_xarray(
               ^^^^^^^^^^^^^
File "/var/lib/buildkite-agent/builds/etl-build-2-15/our-world-in-data/etl-build-public-datasets-branch/.venv/lib/python3.11/site-packages/rioxarray/raster_array.py", line 214, in _clip_xarray
  cropped_ds = xds.where(clip_mask_xray)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
File "/var/lib/buildkite-agent/builds/etl-build-2-15/our-world-in-data/etl-build-public-datasets-branch/.venv/lib/python3.11/site-packages/xarray/core/common.py", line 1189, in where
  return ops.where_method(self, cond, other)

numpy.core._exceptions._ArrayMemoryError: Unable to allocate 3.90 GiB for an array with shape (1009, 721, 1440) and data type float32

To my knowledge there's no memory limit for our Buildkite agents, only what we set with prlimit.
I think there's two approaches that we can do for high memory cases in general:

Pre-reduce the snapshot. We can run a script manually on a high-memory machine to reduce the data to a manageable size, then snapshot the reduced data instead.

If we want to, we can snapshot the both the before and after, but only the reduced after version is actually in our ETL run.

Use out-of-core processing. We can use an out-of-core technique for processing the data, one that still works with limited memory

Databases like DuckDB and even Sqlite3 may do more disk-based work, and thus use less memory
A data frame library like Polars is not out-of-core, but it's more efficient than Pandas since it does query optimisation (NOTE: probably not useful for this specific case)
Vaex advertises itself specifically for the out-of-core use case, it appears to do query optimisation like Polars but mmap the file so it's more flexible about paging memory in and out

Hey, I've just noticed that on my local machine, the meadow step peaks at 35.7GB or RAM! Then I ran it again with gc.collect() right after the clip operation, and memory peaked at 25.9GB. I'd need to repeat the experiment more systematically, but at least that's a possible improvement.

I have also tried clip = da.rio.clip(data.geometry.apply(mapping), data.crs, from_disk=True), since from_disk is "beneficial when the size of the data is larger than memory". I haven't noticed much difference, but it may help in production.

Another way to free up some memory is to delete objects when they are not needed anymore.
    ...
    # Select the 't2m' variable from the combined dataset and assign it to 'da'.
    # Convert the temperature values from Kelvin to Celsius by subtracting 273.15.
    da = ERA5_combine["t2m"] - 273.15
    # Free up memory.
    del ds, ERA5_combine
    ...

Got fixed by https://github.com/owid/etl/pull/2335. If we still experience problems (e.g. with loading that huge gzip file) we should go with the snapshot approach.


Fixed #35742 -- Used USERNAME_FIELD's verbose name in auth's add_form template.

Trac ticket number
ticket-35742
Branch description
On the UserAdmin Add page, I modified the USERNAME_FIELD value to be used for add_form.html
  {% if not is_popup %}

{% translate 'First, enter a username and password. Then, you’ll be able to edit more user options.' %}

  {% else %}

{% translate "Enter a username and password." %}

  {% endif %}

Checklist

[x] This PR targets the main branch.
[x] The commit message is written in past tense, mentions the ticket number, and ends with a period.
[x] I have checked the "Has patch" ticket flag in the Trac system.
[x] I have added or updated relevant tests.
[x] I have added or updated relevant docs, including release notes if applicable.
[ ] I have attached screenshots in both light and dark modes for any UI changes.

We may also have to casefold the username_field_name.

@cliff688
Thank you for your advice on casefold.
I think so too, casefold is also a part that needs to be added.
    def _add_view(self, request, form_url="", extra_context=None):
        ...
        defaults = {
            "auto_populated_fields": (),
            "username_help_text": username_field.help_text,
            "username_field_name": username_field.verbose_name.casefold(),
        }
        ...

How about adding a test code to this file? --> (tests/auth_tests/test_templates.py)
Should I add a test code for changes to the method in add_view as well?

I found a good example.
On the Django Admin login page, the message that appears when the login fails uses USERNAME_FIELD.
class AuthenticationForm(forms.Form):
    def __init__(self, request=None, *args, **kwargs):
        ...
        self.username_field = UserModel._meta.get_field(UserModel.USERNAME_FIELD)

    def get_invalid_login_error(self):
        return ValidationError(
            self.error_messages["invalid_login"],
            code="invalid_login",
            params={"username": self.username_field.verbose_name},
        )

The params dictionary username value delivered when an error occurs is the same as the username_name_field I added to the _add_view method. (Access the verbose_name attribute to the same object.)
And I found one more problem.

I confirmed that the translate is not applied.
I think, I need to modify the pofile of all languages for this part.
django/contrib/admin/locale/all/LC_MESSAGES/django.po
...
msgid ""
"First, enter a %(username_name_field)s and password. Then, you’ll be able to edit more user "
"options."
msgstr ""
"첫 번째로, %(username_name_field)s과 비밀번호를 입력하세요. 그 후, 독자는 더 많은 사용자 옵"
"션을 수정할 수 있습니다. "
...

How about modifying username_field_name to username for consistency?
It doesn't seem to cause problems even if it's used as a username key.
def _add_view(self, request, form_url="", extra_context=None):
        defaults = {
            "auto_populated_fields": (),
            "username_help_text": username_field.help_text,
            # "username_field_name": username_field.verbose_name,
            "username": username_field.verbose_name,
        }

@cliff688 Thank you for your advice on casefold. I think so too, casefold is also a part that needs to be added.

Actually I found out in the docs that

The convention is not to capitalize the first letter of the verbose_name. Django will automatically capitalize the first letter where it needs to.

So we don't have to casefold the name

How about modifying username_field_name to username for consistency?
It doesn't seem to cause problems even if it's used as a username key.

I don't see any problem with that.

I have updated all the .po files so that the username template variable can be translated.
Only the part that was previously used as 'username' was changed to %(username)s, but some languages were unable to change the 'username' part because it was difficult to distinguish exactly.
What should I do about these parts?

I have updated the release for the changes.
:mod:`django.contrib.admin`
~~~~~~~~~~~~~~~~~~~~~~~~~~~
...
* The ``admin/auth/user/add_form.html`` input guideline message is applied
  with a value according to user model username_field.
...

:mod:`django.contrib.auth`
~~~~~~~~~~~~~~~~~~~~~~~~~~
...
* :meth:`.UserAdmin._add_view` The username_field verbose_name attribute of the
  user model was added as a username key to the defaults dictionary.
...

I don't know if the release was written well. I need a feedback. ✋

@Antoliny0919 Thank you for continuing the work in this PR. :star2:
I will update the ticket and I will set the "patch needs tests" and "patch needs improvement" flag to signal that there is still a need for:

fix the docs building errors, and
add tests for this change.

Looking forward to those fixes!

I have created a test code to test whether username_field is applied to the add_form.html template.
class UserAdminTest(TestCase):
    ...
    def test_guideline_username(self):
        username = User._meta.get_field(User.USERNAME_FIELD).verbose_name
        response = self.client.get(reverse("admin:auth_user_add"))
        self.assertContains(
            response,
            "
First, fill the fields: %s and password. "
            "Then, you’ll be able to edit more user options.
" % (username),
            html=True,
        )
        response = self.client.get(
            reverse("admin:auth_user_add") + "?%s=1" % IS_POPUP_VAR
        )
        self.assertContains(
            response,
            "
Fill the fields: %s and password.
" % (username),
            html=True,
        )
    ...

Test the input guideline messages in the add_form.html template when popup is applied and not.
If there is anything lacking about the test code, I would appreciate it if you could advise me.
I'd appreciate it if you could advise me on the test function name as well.
It was the best thing for my naming sense..

I have reflected all of your suggestions, thank you so much @sarahboyce !

So far, I have found two parts where username has been used.
# admin
"Please enter the correct %(username)s and password for a staff account. Note "
"that both fields may be case-sensitive."

# auth
"Please enter a correct %(username)s and password. Note that both fields may "
"be case-sensitive."

With the above two and the review given by @claudep, I think the following three are candidates to be guidelines messages.
1. First, enter a {{ username }} and password.

2. First, enter the {{ username }} and password.

3. First, fill the fields: {{ username }} and password.

Considering the consistency, it's number 1 and 2, but I think number 3 is also a good choice considering the translation according to the review claudep gave me.
I'm curious about the opinions of members and contributers. What should I use as a guideline message?

I confirmed that it becomes more awkward when I used USERNAME_FIELD only for add_form.html.
As an example, if I change USERNAME_FIELD to email and add_form.html to use the USERNAME_FIELD value, it is as follows.

As above, the changed input guidelines show an email, but the form still has a username.
Therefore, to configure dynamic values through USERNAME_FIELD, we must also use USERNAME_FIELD in UserAdmin's add_fieldset.
@admin.register(User)
class UserAdmin(admin.ModelAdmin):
   ...
    add_fieldsets = (
        (
            None,
            {
                "classes": ("wide",),
                "fields": (User.USERNAME_FIELD, "usable_password", "password1", "password2"),
            },
        ),
    )
    ...

I thought this change was an improvement in UserAdmin provided by Django, so I modified the related issue.

I applied the parts you suggested. @sarahboyce
And I also applied the parts suggested by @nessita on the ticket.
I have one question.
There are some problems that users may experience when using USERNAME_FIELD for UserAdmin in the way I suggested.
Assuming that the user changes USERNAME_FIELD to a different field using a custom user model, the most likely candidate is "email."
If the user changes USERNAME_FIELD to one of the fields provided by Django AbstractUser, the fieldset value of UserAdmin must also be changed.
Because fieldsets return errors for duplicate field values.
For example, when I changed USERNAME_FIELD to "email" and applied UserAdmin.

The above error occurs because the fieldsets do not allow duplicate fields, as I have previously explained.
username = get_user_model().USERNAME_FIELD # --> email

@admin.register(User)
class UserAdmin(admin.ModelAdmin):
    add_form_template = "admin/auth/user/add_form.html"
    change_user_password_template = None
    fieldsets = (
        (None, {"fields": (username --> "email", "password")}),
        (_("Personal info"), {"fields": ("first_name", "last_name", "email")}),
        ...

The list_display in ModelAdmin is also affected and displays duplicate fields.

Of course, removing duplicates of those fields will fix the problem.
However, I thought these parts might rather inconvenience Django users, so I was curious about other people's opinions.

Assuming that the user changes USERNAME_FIELD to a different field using a custom user model, the most likely candidate is "email."

Agreed, although email needs to be overridden to be a unique field and won't work out of the box.
The other option is a new field

If the user changes USERNAME_FIELD to one of the fields provided by Django AbstractUser, the fieldset value of UserAdmin must also be changed.

I don't think there is a perfect solution given that the user has to be overriding the model.
The minimal change is doing just the templates, as those are the hardest to override.
I wouldn't worry about the list and search fields - they still "work"
The main thing for me is can a User be created and updated? I think the rest of the work should be done as part of ticket-28608 in a separate PR

I don't think there is a perfect solution given that the user has to be overriding the model.
The minimal change is doing just the templates, as those are the hardest to override.

Yes, I agree with you.
If so, is there any more work I have left in this PR?
Please let me know if there is any more work left and I will proceed.


about the background categories and Instances and Tasks

Hello, thank you very much for your work. In EQL, the cross entropy loss is used in the calculation of background categories. In eqlv2, how to deal with the background category.
and
How to try to understand this sentence ：“one task with positive label and C−1 tasks with negative labels are introduced by a single instance.”
look forward to your reply ！

Suppose we have C=5 categories.
In softmax loss, the classifier output c+1 channel, the extra 1 is for background. In inference,  the background scores will suppress other categories' scores since we have a softmax operation.
In sigmoid loss, the classifier output c  channel.  Each channel is then activated by a sigmoid function and responsible for one category. So each channel needs a gt_label (0, 1) in training.
Given a proposal of category j, for example j=2, we re-write the gt_label to [0, 0, 1, 0, 0]. That is: one instance introduces one task with positive label and C−1 tasks with negative labels.
For background task, the gt_label is [0, 0, 0, 0, 0]

I'm very sorry to reply so late. I understand this question. Thank you very much for your answer! thanks again!


NTR modulation by symbiont of host system process

Hello,
The multiorganism working group proposes to create a new term, 'modulation by symbiont of host system process ', to organize the 'modulation by symbiont of host process' branch, which would group:
'modulation by symbiont of host digestive system process'
'modulation by symbiont of host endocrine process'
'modulation by symbiont of host nervous system process'
'modulation by symbiont of host respiratory system process'
+[Term]
+id: GO:0140780
+name: modulation by symbiont of host system process
+namespace: biological_process
+def: "The process in which a symbiont organism effects a change in an anatomical system process of its host organism." [GOC:pg]
+intersection_of: GO:0044003 ! modulation by symbiont of host process
+intersection_of: regulates GO:0003008 ! system process
+created_by: pg
+creation_date: 2022-03-28T09:54:08Z

Rehoused'orphan' term that did not fall under

'modulation by symbiont of host cellular process'
'modulation by symbiont of host system process'

Moved under 'modulation by symbiont of host cellular process':

'modulation by virus of host molecular function'
induction by symbiont of tumor or growth in host

Moved under 'modulation by virus of host cellular process':

transformation of host cell by virus

GO:0019087 modulation by virus of host immune response: moved from GO:0019048 ! modulation by virus of host process to is_a: GO:0019054 ! modulation by virus of host cellular process

GO:0039599 cleavage by virus of host mRNA 0 annotations:  merge into GO:0039604 suppression by virus of host translation

GO:0039600  induction by virus of host endonucleolytic cleavage-dependent mRNA catabolic process: 0 annotations: merge into GO:0039604 suppression by virus of host translation

At a higher level:

GO:0044003 modulation by symbiont of host process: add subclass: 'modulation of process of another organism'