Add feature flag and render custom paths behind feature flag

Add feature preview functionaility;
Add custom paths rendering behind new feature preview functionaility;
Small fix re. player position rendering while in compass mode and panned;

Screenshots:
Feature flag page

After enabling the flag

Inputting an example geojson blob near monach's bluff [1]

Path shown as two distinct segments

Doesn't interfere with nav

Nor does it interfere with text

Line size do not respect icon size by deisgn

[1] - geojson blob near monach's bluff
{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {
        "legend": "Testing",
        "strokeStyle": "#EBA5AB",
        "lineWidth": 10
      },
      "geometry": {
        "type": "LineString",
        "coordinates": [
          [
            7307,3608

          ],
          [
            7390,3597

          ],
          [
            7420,3587

          ],
          [
            7417,3550

          ]
        ]
      }
    },
    {
      "type": "Feature",
      "properties": {
        "legend": "Testing",
        "strokeStyle": "#B3C8EA",
        "lineWidth": 5
      },
      "geometry": {
        "type": "LineString",
        "coordinates": [
          [
            7407,3608

          ],
          [
            7490,3597

          ],
          [
            7520,3587

          ],
          [
            7517,3550
          ]
        ]
      }
    }
  ]
}

I'm not that familiar with geojson. But what benefits does it provide over having a route system where routes are stored as a list of points and an indication of how this point should be reached from the previous point (e.g. straight line or using the navigation mesh)? (ignoring the line color here)
I understand that there are some extra features that this could provide, but I fear that this makes things slightly harder to manage (both code-wise and for users themselves). I also think this limits the possibility (without having to do workarounds) of using the navigation mesh for finding a path for a given route.

GeoJSON is just a canonical way to representing geographical features (points, lines, polygon and whatnot) along with its associated non-spatial attributes (this can be whatever is relevant, e.g. color, title, other styling for rendering, etc.).
We can totally go for just array of points as representation, though as soon as you add any metadata to this array of points, it becomes effectively a homebrewed version of GeoJSON in my opinion, using GeoJSON from the get-go seemed more sensible to me.
Assuming you just want an array of points for manipulation, computing this from the supplied GeoJSON should be relatively easy anyway.

But then I would like to know how you believe we can add functionality to this which utilises the navigation mesh. We can just get the list of points for that part and add those, but adding hundreds of nodes to the geojson data might not be so super nice when manually editing it. Do you have any suggestions for that?

add functionality to this which utilises the navigation mesh
Could you clarify what you are trying to achieve?
I probably misunderstood what you said and hence couldn't see why array of points would be preferable.

Currently there is the navigation mesh feature (middle-mouse button on the map by default). I think it could be useful when defining custom routes if a user could for example select like 5 points and create a route past those 5 points using the navigation system already implemented (instead of straight lines). It would be nice if this worked together with all other map drawing things (so a line following known roads instead of a straight line). But I'm not entirely sure if or how it could work together.
One way would be to mark 2 locations in the UI, generate the path using navigation and storing all individual nodes along that road and storing them in the geojson format, but this might add a very large number of nodes to the route, which could make manual editing very cumbersome.
I know your PR doesn't contain UI for creating the geojson data, so perhaps all of this is not fully relevant yet.
@pvdstel your thoughts?

Ah I see what you are trying to do now.
One possible solution is to render different types of feature type in GeoJSON differently, if feature === LineString then draw naive straight line as is right now, else if feature === MultiPoint then draw dots on the map with an added option to join the dots with the navigation mesh. Does that make sense?
For refernce, a LineString feature looks like
{
    "type": "LineString",
    "coordinates": [
        [30.0, 10.0], [10.0, 30.0], [40.0, 40.0]
    ]
}

and a MultiPoint feature looks like
{
    "type": "MultiPoint",
    "coordinates": [
        [10.0, 40.0], [40.0, 30.0], [20.0, 20.0], [30.0, 10.0]
    ]
}

Almost the same thing but not quite.

Here is a quick POC

The pink line is the "ground truth" consists of only 4 points.
The blue-ish line is generated by taking these 4 points and calling findPath in a piecewise fashion and joining them all together.
It's a little wonky but I think it is expected given the mesh looks like this

For reference, here is the GeoJSON blob used,
{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {
        "legend": "Testing",
        "strokeStyle": "#EBA5AB",
        "lineWidth": 10
      },
      "geometry": {
        "type": "LineString",
        "coordinates": [
          [
            7436,
            3514
          ],
          [
            7486,
            3595
          ],
          [
            7422,
            3592
          ],
          [
            7483,
            3758
          ]
        ]
      }
    },
    {
      "type": "Feature",
      "properties": {
        "legend": "Testing",
        "strokeStyle": "#B3C8EA",
        "lineWidth": 5
      },
      "geometry": {
        "type": "MultiPoint",
        "coordinates": [
          [
            7436,
            3514
          ],
          [
            7486,
            3595
          ],
          [
            7422,
            3592
          ],
          [
            7483,
            3758
          ]
        ]
      }
    }
  ]
}


[Bilibili] 互动视频 interactive split-path videos

DO NOT REMOVE OR SKIP THE ISSUE TEMPLATE

[X] I understand that I will be blocked if I remove or skip any mandatory* field

Checklist

[X] I'm reporting a broken site
[X] I've verified that I'm running yt-dlp version 2023.01.06 (update instructions) or later (specify commit)
[X] I've checked that all provided URLs are playable in a browser with the same IP and same login details
[X] I've checked that all URLs and arguments with special characters are properly quoted or escaped
[X] I've searched the bugtracker for similar issues including closed ones. DO NOT post duplicates
[X] I've read the guidelines for opening an issue
[X] I've read about sharing account credentials and I'm willing to share it if required

Region
Singapore
Provide a description that is worded well enough to be understood
https://www.bilibili.com/video/BV14Q4y1q7nD
There are videos like these known in Chinese as 互动视频. Basically, they're interactive videos where you get to pick your own path through the videos. Usually they're like a few minutes long, then a option for 2 or more split paths.
Currently, yt-dlp does not appear to support this at all. The sub-paths do not have unique or independent urls, so its quite impossible to download them. Nor is it feasible to do it manually, if there are say 5 levels and 2^5 is quite tedious.
What currently happens is, it downloads 1st level, and stops. It doesn't iterate through all of the sub-levels.
Provide verbose output that clearly demonstrates the problem

[X] Run your yt-dlp command with -vU flag added (yt-dlp -vU <your command line>)
[X] Copy the WHOLE output (starting with [debug] Command-line config) and insert it below

Complete Verbose Output
yt-dlp -vU https://www.bilibili.com/video/BV14Q4y1q7nD
[debug] Command-line config: ['-vU', 'https://www.bilibili.com/video/BV14Q4y1q7nD']
[debug] User config "C:\Users\Pxssy\AppData\Roaming\yt-dlp\config": ['--cookies', 'C:/Users/Daryl/AppData/Roaming/yt-dlp/cookies', '--output', './yt-dlp/%(uploader,channel)s (%(uploader_id,channel_id)s)/[%(upload_date)s] %(title).150s [%(resolution)s@%(fps)sp] (%(id)s).%(format_id)s.%(ext)s', '--format', 'bestvideo+bestaudio/best', '--format-sort', 'res,fps,quality,size,br', '--retries', 'infinite', '--fragment-retries', 'infinite', '--ignore-errors', '--mtime', '--no-continue', '--no-overwrites', '--part', '--write-thumbnail', '--write-annotations', '--write-auto-subs', '--write-description', '--write-info-json', '--write-playlist-metafiles', '--write-subs']
[debug] Encodings: locale cp1252, fs utf-8, pref cp1252, out utf-8, error utf-8, screen utf-8
[debug] yt-dlp version 2023.01.06 [6becd25] (pip)
[debug] Python 3.9.6 (CPython AMD64 64bit) - Windows-10-10.0.19044-SP0 (OpenSSL 1.1.1k  25 Mar 2021)
[debug] exe versions: ffmpeg 4.2.3, ffprobe 5.0.1-full_build-www.gyan.dev
[debug] Optional libraries: Cryptodome-3.11.0, brotli-1.0.9, certifi-2021.10.08, mutagen-1.45.1, sqlite3-2.6.0, websockets-9.1
[debug] Proxy map: {}
[debug] Loaded 1760 extractors
[debug] Fetching release info: https://api.github.com/repos/yt-dlp/yt-dlp/releases/latest
Latest version: 2023.01.06, Current version: 2023.01.06
yt-dlp is up to date (2023.01.06)
[BiliBili] Extracting URL: https://www.bilibili.com/video/BV14Q4y1q7nD
[BiliBili] 14Q4y1q7nD: Downloading webpage
[BiliBili] BV14Q4y1q7nD: Extracting videos in anthology
[BiliBili] Format(s) 1080P 高码率, 1080P 高清, 720P 高清 are missing; you have to login or become premium member to download them. Use --cookies-from-browser or --cookies for the authentication. See  https://github.com/yt-dlp/yt-dlp/wiki/FAQ#how-do-i-pass-cookies-to-yt-dlp  for how to manually pass cookies
[BiliBili] 721319002: Extracting chapters
[BiliBili] BV14Q4y1q7nD: Downloading JSON metadata
[info] BV14Q4y1q7nD: Downloading subtitles: danmaku
[debug] Sort order given by user: res, fps, quality, size, br
[debug] Formats sorted by: hasvid, ie_pref, res, fps, quality, filesize, fs_approx, tbr, vbr, abr, lang, hdr:12(7), vcodec:vp9.2(10), channels, acodec, asr, proto, vext, aext, hasaud, source, id
[info] BV14Q4y1q7nD: Downloading 1 format(s): 8+2
[info] Video description is already present
[info] Video subtitle danmaku.xml is already present
[info] Video thumbnail is already present
[info] Video metadata is already present
WARNING: There are no annotations to write.
[download] yt-dlp\绝不早到小吱吱 (11616487)\[20211028] 【互动】当你的女房东是个间谍，你会……<EMAIL_ADDRESS>(BV14Q4y1q7nD).8+2.mp4 has already been downloaded

互动视频是没法这样下载的吧...是特殊格式

I have tried writing a path-extractor to traverse all divisions. Each piece of video segment is identified with a unique cid (paired with aid) to locate video resources. The regular video link now serves as a playlist link instead.
However, it may require some major change to use aid-cid pair instead of url as minimal unit to get video. It looks hard to me to download an interactive-video link as a playlist link without refactoring the main code block.
Here are the code for getting cid of video segments:

class BilibiliBaseIE(InfoExtractor):
...
    def _get_divisions(self, aid, graph_version, edges={}, edge_id=""):
        division_data = self._download_json(
            f'https://api.bilibili.com/x/stein/edgeinfo_v2?graph_version={graph_version}&edge_id={edge_id}&aid={aid}',
            aid, note=f'Extracting divisions from edge {edge_id}', fatal=True)
        # extracting info from history nodes
        for node in traverse_obj(division_data, ('data', 'story_list'), default=[]):
            edges[node['edge_id']] = traverse_obj(node, {
                'title': ('title', {str_or_none}),
                'cid': ('cid', {int}),
            })
        # extracting info from current node and upcoming nodes
        edges[edge_id] = {
            **edges.get(edge_id, {}),
            'title': str_or_none(traverse_obj(division_data, ("data", "title"))),
            'choices': traverse_obj(division_data, ("data", "edges", "questions", ..., "choices", ..., {
                'edge_id': ('id', {int}),
                'cid': ('cid', {int}),
                'text': ('option', {str_or_none}),
            })),
        }
        # traverse choices and following nodes
        for choice in edges[edge_id]['choices']:
            if choice['edge_id'] not in edges:
                edges[choice['edge_id']] = {
                    'cid': choice['cid'],
                }
                self._get_divisions(aid, graph_version, edges=edges, edge_id=choice['edge_id'])
        return edges

class BiliBiliIE(BilibiliBaseIE):
...

    def _real_extract(self, url):
...

        is_interactive = traverse_obj(video_data, ("rights","is_stein_gate"))
        if is_interactive:
            graph_version = traverse_obj(
                self._download_json(
                    f'https://api.bilibili.com/x/player/wbi/v2?aid={aid}&cid={cid}',
                    aid, note=f'Extracting graph version for av{aid}', fatal=True),
                ('data', 'interaction', 'graph_version'))
            divisions = self._get_divisions(aid, graph_version, {1: {"cid": cid}}, 1)
            # TO DO: download each segment using aid-cid pair

@pxssy I've written a patch to download interactive videos as playlists. It would be nice if you can test this branch and provide some feedbacks:
https://github.com/c-basalt/yt-dlp/tree/bili-interactive

https://www.ithome.com/0/694/102.htm

Also related to BilibiliIE so leave the memo here: code 87007 ("message":"87007") for https://api.bilibili.com/x/player/playurl of playinfo is for member-only video. Need to change playinfo = ... ['data'] to playinfo = ... .get('data') and hint login
Additional info can be found at initial_state['elecFullInfo']['show_info']. Message can be obtained by clean_html(get_element_by_class('tips-toast', webpage))


Withdraw Ignis

I purchased your pre-sold IGNIS with the promise of being able to withdraw shortly (<1 month) after the chain launch. Now I can't even get a dev response. Can you please add support for this today or exchange to me directly today OTC?

Please contact the asset issuer.
This issue tracker is meant for issues relating to the heat-ui codebase.
Closing this issue.

Isn't heat the asset issuer?


Tile archiving: consistency check between folders and tiles-specstatus.ecsv

Following today s discussed topic, if useful, here are few lines of code which should catch any inconsistency, for the mainbright and maindark tiles, between:

what s in $DESI_ROOT/spectro/redux/daily/tiles/archive
what s reported as archived in $DESI_SURVEYOPS/ops/tiles-specstatus.ecsv.

We could for instance plug that at the beginning of the run_mtl_loop script? or else..
I ve tested those lines with the current files, and it says everything is fine; it runs in few seconds.
I could mock up later some fake data to verify it breaks if it should -- i.e. that I didn t mess up the coding.
any comment / suggestion is welcome!
import os
from glob import glob
from astropy.table import Table
from desiutil.log import get_logger

log = get_logger()

# what s in the archive
archivedir = "/global/cfs/cdirs/desi/spectro/redux/daily/tiles/archive"
fns = []
for pattern in ["????", "1????", "2????", "3????"]:
    fns += sorted(glob(os.path.join(archivedir, pattern, "*")))

fns = [fn for fn in fns if os.path.basename(fn) != "20210623.try1"]
arch_d = Table()
arch_d["TILEID"] = [int(fn.split(os.path.sep)[-2]) for fn in fns]
arch_d["ARCHIVEDATE"] = [int(fn.split(os.path.sep)[-1]) for fn in fns]
arch_d["TILEID_ARCHDATE"] = ["{}-{}".format(tileid, archdate) for tileid, archdate in zip(arch_d["TILEID"], arch_d["ARCHIVEDATE"])]
# in case of multiple ARCHIVEDATE for a TILEID, only keep the latest ARCHIVEDATE
ii = np.lexsort((-arch_d["ARCHIVEDATE"], arch_d["TILEID"]))
arch_d = arch_d[ii]
_, ii = np.unique(arch_d["TILEID"], return_index=True)
arch_d = arch_d[ii]

# what s in tiles-specstatus.escv
fn = os.path.join(os.getenv("DESI_SURVEYOPS"), "ops", "tiles-specstatus.ecsv")
svn_d = Table.read(fn)
svn_d["TILEID_ARCHDATE"] = ["{}-{}".format(tileid, archdate) for tileid, archdate in zip(svn_d["TILEID"], svn_d["ARCHIVEDATE"])]
sel = (svn_d["FAFLAVOR"] == "mainbright") | (svn_d["FAFLAVOR"] == "maindark")
sel &= svn_d["ARCHIVEDATE"] != 0
svn_d = svn_d[sel]

# tiles in archive, not in tiles-specstatus
arch_sel = ~np.in1d(arch_d["TILEID_ARCHDATE"], svn_d["TILEID_ARCHDATE"])
arch_msg = "nb of tiles in archive, not in tiles-specstatus: {}".format(arch_sel.sum())
# tiles in tiles-specstatus, not in archive
svn_sel = ~np.in1d(svn_d["TILEID_ARCHDATE"], arch_d["TILEID_ARCHDATE"])
svn_msg = "nb of tiles in archive, not in tiles-specstatus: {}".format(svn_sel.sum())

if (arch_sel.sum() == 0) & (svn_sel.sum() == 0):
    log.info(arch_msg)
    log.info(svn_msg)
else:
    if arch_sel.sum() > 0:
        print(arch_d[arch_sel].pprint_all())
        log.error(arch_msg)
    if svn_sel.sum() > 0:
        print(svn_d[svn_sel].pprint_all())
        log.error(svn_msg)
    raise ValueError(arch_msg + " ; " + svn_msg)

Sure, I can add something like this to run_mtl_loop. I'll assign myself in a desitarget issue soon.

thanks @schlafly for the comments.

"you could probably handle the weird file name case more cleanly by making your pattern '????????' instead of '*'." => indeed, yes, thanks, that looks better!
"you've excluded SV3, backup, and engineering tiles by ~tileid. Another approach would be to exclude any tileid not in tiles-main.ecsv, which would remove the SV3 and engineering tiles, but not the backup tiles. You could remove those, too, by additionally putting in a PROGRAM != 'BACKUP' check." => yes, that should also work; and indeed may be more robust to future changes -- or cleaner;
"I imagine you've been careful here because there are some archived BACKUP tiles, but no MTL updates for those archived backup tiles?" => kind of; I thought that, as only main{bright,dark} tiles are updated in the MTL ledgers, that s the only set of tiles we want to check; but I think nothing prevents to check all tiles, if we wanted to; the risk being to divert the attention of the people doing the dailyops with non-relevant tiles.

if useful, for reference, looking a bit more closely at non-main{bright,dark} tiles, I see that:

488 sv3 tiles are archived, and consistently  (ie the same number of sv3 tiles) have ARCHIVEDATE!=0 -- ok;
2 mainbackup tiles are archived,  and consistently have ARCHIVEDATE!=0: 41712, 41713 -- ok;
26 TILEID > 80000 tiles have ARCHIVEDATE!=0, but are actually not archived; those all are sv2 tiles, except one (TILEID=82264) -- this is inconsistent, but we probably don t care...

and a last remark: note that in my approach, I just check the folder existence, not its content.
I feel it s a safe enough check, and checking individual files may add complexity+computing time, for a little gain; but I mention in case people think we d want to do a per-file check.

FWIW, it looks like 82264 was our first special test backup tile.  I suspect this relates to when we started archiving and what we decided to archive then.  It sounds like requiring the TILEID to be in tiles-main would safely exclude these cases and is probably the most understandable approach.

@sbailey , should we get rid of daily/tiles/archive/1000/20210623.try1? I don't remember what that was.

That appears to be a leftover from early tile archive testing; I removed it.

I think we can close this?

as far as I know, yes, as it s now merged in desitarget.
just, to be sure: when running the mtl update with desiproc, it s desitarget/main which is loaded by the gomtl command, not some tagged version?

Good point!  No, 2.5.0 for MTL updates.
gomtl ()
{
#    source /global/project/projectdirs/desi/software/desi_environment.sh 21.7e
# moving to 22.5 for compatibility with perlmutter
    source /global/common/software/desi/desi_environment.sh 22.5
    module swap desitarget/2.5.0
    export MTL_DIR=$SURVEYOPS/mtl
}

Obviously very intentionally to avoid doing MTL updates with different versions of things.  @geordie666 , how concerned do we need to be about updating the desitarget version used for MTL updates?

I think we should definitely make a new tag and run with what will then be desitarget 2.7.0.
I'd want a little time to do that though, to check we aren't going to immediately break anything. And I'm swamped today. Maybe right after Thanksgiving to avoid breaking anything close to Thanksgiving?

Good.  Yes.  After Thanksigiving makes sense; I'll leave this issue open.

I think we're all done here and can close?

I confirm the gomtl command now uses the latest desitarget/2.7.0 tag.
I m closing this issue.
gomtl ()
{
#    source /global/project/projectdirs/desi/software/desi_environment.sh 21.7e
# moving to 22.5 for compatibility with perlmutter
#    source /global/common/software/desi/desi_environment.sh 22.5
# moving to 23.10 for compatibility with desitarget/2.7.0
    source /global/common/software/desi/desi_environment.sh 23.10
    module swap desitarget/2.7.0
    export MTL_DIR=$SURVEYOPS/mtl
}


Coin order on update

When a user already has a set of coins selected/saved and then adds another, the existing custom order is sometimes messed up.
Need to figure out how to just put the latest coin on either the top or bottom.

Fixed, default sortorder for newly added coins is 999, which should put them at the bottom by default until the user changes the order.
Will be included in the next release.


The consumption performance index cannot be obtained, and the obtained value is all 0

Describe the bug
The consumption performance index cannot be obtained, and the obtained value is all 0
To Reproduce
Steps to reproduce the behavior:

Send and consume messages
Obtain indicator information through the indicator interface  http://ip:8080/metrics/
Observe the pulsar_rate_out indicator information

Additional context:
version：2.5.2
class:NamespaceStatsAggregator
method:getTopicStats

Closed as stale. Please create a new issue if it's still relevant to the maintained versions.


WorkloadIdentityCredential missing in DefaultAzureCredential

Package Name:  azure-identity
Package Version:  1.16.0
Operating System: Linux
Python Version:  3.12.3

Describe the bug
According to the documentation WorkloadIdentityCredential is included on the Default Credential  which is not.
To Reproduce
Steps to reproduce the behavior:

Given the following env vars:

AZURE_CLIENT_ID
AZURE_TENANT_ID
AZURE_FEDERATED_TOKEN_FILE

The following code fails to obtain a valid credential
credential = WorkloadIdentityCredential()

And the following does
credential = WorkloadIdentityCredential()

Expected behavior
I would expect that if the exclude_workload_identity_credential is not provided, Workload identity should work with Default Credential.
Screenshots
Stacktrace upon failing:
	EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
	ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource. Error: Unexpected response "{'error': 'invalid_request', 'error_description': 'Identity not found'}"
	SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
	AzureCliCredential: Azure CLI not found on path
	AzurePowerShellCredential: PowerShell is not installed
	AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.
Traceback (most recent call last):
  File "/opt/docker/generate-template.py", line 41, in <module>
Processing templates for mx-datagendbflex-acc1-kyc-westeurope...
    instance['client_keys'] = get_secrets(instance['client_keys'])
  File "/opt/docker/generate-template.py", line 16, in get_secrets
    res.update({user: client.get_secret(secret).value})
  File "/usr/local/lib/python3.10/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/azure/keyvault/secrets/_client.py", line 64, in get_secret
    bundle = self._client.get_secret(
  File "/usr/local/lib/python3.10/site-packages/azure/core/tracing/decorator.py", line 78, in wrapper_use_tracer
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/azure/keyvault/secrets/_generated/_operations/_operations.py", line 751, in get_secret
    pipeline_response: PipelineResponse = self._client._pipeline.run(  # pylint: disable=protected-access
  File "/usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py", line 230, in run
    return first_node.send(pipeline_request)
  File "/usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py", line 86, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py", line 86, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.10/site-packages/azure/core/pipeline/_base.py", line 86, in send
    response = self.next.send(request)
  [Previous line repeated 2 more times]
  File "/usr/local/lib/python3.10/site-packages/azure/core/pipeline/policies/_redirect.py", line 197, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.10/site-packages/azure/core/pipeline/policies/_retry.py", line 531, in send
    response = self.next.send(request)
  File "/usr/local/lib/python3.10/site-packages/azure/core/pipeline/policies/_authentication.py", line 135, in send
    request_authorized = self.on_challenge(request, response)
  File "/usr/local/lib/python3.10/site-packages/azure/keyvault/secrets/_shared/challenge_auth_policy.py", line 137, in on_challenge
    self.authorize_request(request, scope, tenant_id=challenge.tenant_id)
  File "/usr/local/lib/python3.10/site-packages/azure/core/pipeline/policies/_authentication.py", line 113, in authorize_request
    self._token = self._credential.get_token(*scopes, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/azure/identity/_credentials/default.py", line 225, in get_token
    token = super().get_token(*scopes, claims=claims, tenant_id=tenant_id, **kwargs)
  File "/usr/local/lib/python3.10/site-packages/azure/identity/_credentials/chained.py", line 124, in get_token
    raise ClientAuthenticationError(message=message)
azure.core.exceptions.ClientAuthenticationError: DefaultAzureCredential failed to retrieve a token from the included credentials.
Attempted credentials:
	EnvironmentCredential: EnvironmentCredential authentication unavailable. Environment variables are not fully configured.
Visit https://aka.ms/azsdk/python/identity/environmentcredential/troubleshoot to troubleshoot this issue.
	ManagedIdentityCredential: ManagedIdentityCredential authentication unavailable. The requested identity has not been assigned to this resource. Error: Unexpected response "{'error': 'invalid_request', 'error_description': 'Identity not found'}"
	SharedTokenCacheCredential: SharedTokenCacheCredential authentication unavailable. No accounts were found in the cache.
	AzureCliCredential: Azure CLI not found on path
	AzurePowerShellCredential: PowerShell is not installed
	AzureDeveloperCliCredential: Azure Developer CLI could not be found. Please visit https://aka.ms/azure-dev for installation instructions and then,once installed, authenticate to your Azure account using 'azd auth login'.
To mitigate this issue, please refer to the troubleshooting guidelines here at https://aka.ms/azsdk/python/identity/defaultazurecredential/troubleshoot.

Additional context
For anyone trying to setup Workload Identity with Azure pipelines, this setup with which I managed to perform login on an Azure Devops Task pipeline:
  - task: AzureCLI@2
    displayName: "Generate templates"
    inputs:
      addSpnToEnvironment: true
      azureSubscription: ${{ parameters.sc }}
      scriptType: bash
      scriptLocation: inlineScript
      inlineScript: |
        echo "$idToken" > /shared-data/tokenfile.json
        docker run --rm \
          -v /shared-data/tokenfile.json:/opt/docker/tokenfile.json  \
          -e "AZURE_CLIENT_ID=$servicePrincipalId" \
          -e "AZURE_TENANT_ID=$tenantId" \
          -e "AZURE_FEDERATED_TOKEN_FILE=/opt/docker/tokenfile.json" \
          mytemplategeneratorimage:${{ parameters.generatorVersion }}

and on my mytemplategeneratorimage the following python code.
credential = WorkloadIdentityCredential()

Although it seems they have recently implemented #32143 which would also work (but also is not included in the DefaultAzureCredential helper) but that is not yet released nor the documentation updated

Thanks for reaching out.
In order to make WorkloadIdentity in DefaultAzureCredential, please also set the env var AZURE_AUTHORITY_HOST .


handling of shock located files in DIffExp viewer

Sample files above a certain size are getting placed on shock. The Experiment.js viewer did not expect this.

I committed a fix for this in https://github.com/dmachi/p3-web/commit/37de53e6bf4bfb09ab1bf32c7e7ef1b677e050d6  , though i've just noticed i committed to the djm branch instead of master

Dustin's fix has been deployed.


Add unit tests for triggering (or not) table existence checks

A few additional tests cases that checks if table existence checks are triggered only when autoCreateTable is set to false and they fail in case of a missing tables.
This PR depends on #238

Travis test failures are caused by missing symbols that were introduced in #238
Once that PR will be merged they will become green.


audit rules template mistake - fdisk

Describe the Issue
A path mistake in the audit rules template for Ubuntu 20 causes the audit fdisk check to fail when benchmarking.
The problematic line is https://github.com/ansible-lockdown/UBUNTU20-STIG/blob/0e94b5de1439c772d320d77805bbe05f106f197b/templates/audit/99_stig_auditd.rules.j2#L148
The benchmark check correctly paths fdisk to /sbin/fdisk:
  <ind:pattern operation="pattern match">^[\s]*-w[\s]+/sbin/fdisk[\s]+-p[\s]+x[\s]+-k[\s]+modules[\s]*$</ind:pattern>

Expected Behavior
Check should pass
Actual Behavior
Check does not pass as the rule is created for /bin/fdisk and not /sbin/fdisk
Control(s) Affected
UBTU-20-010298
Environment (please complete the following information):

branch being used: [e.g. devel]
Ansible Version: [e.g. 2.10]
Host Python Version: [e.g. Python 3.7.6]
Ansible Server Python Version: [e.g. Python 3.7.6]
Additional Details:

Additional Notes
Anything additional goes here
Possible Solution
Fix template at line number 148 as permalinked above.
s/bin/sbin

hi @kfiresmith
Thank you for raising this issue, I am happy to say that the suggested fix has been added and merged to both devel and main branches.
Unless you have any issue, i believe this could now be closed?
many thanks
uk-bolly


-vb duplicates progress bar output

Hi,
if you run vmtouch -vb the progress bar lines are added twice to the output file. It would be great if the lines just occur once.
Steps to reproduce:
# fgrep -r "foobarHURZ" /usr/

# find /usr/ -type f | vmtouch -vb - > vmtouch.out

# less vmtouch.out
/usr/lib/libxcb-xv.so.0.0.0
^M[OOOOOOOO] 8/8^M[OOOOOOOO] 8/8
/usr/lib/libXext.so.6.4.0
^M[OOOOOOOOOOOOOOOOOOO] 19/19^M[OOOOOOOOOOOOOOOOOOO] 19/19
/usr/lib/libX11-xcb.so.1.0.0
^M[OO] 2/2^M[OO] 2/2
[...]

# cat vmtouch.out | tr '\r' '\n'
/usr/lib/libxcb-xv.so.0.0.0

[OOOOOOOO] 8/8
[OOOOOOOO] 8/8
/usr/lib/libXext.so.6.4.0

[OOOOOOOOOOOOOOOOOOO] 19/19
[OOOOOOOOOOOOOOOOOOO] 19/19
/usr/lib/libX11-xcb.so.1.0.0
[...]

Regards,
Carstem

Hi,
The reason you're seeing two is that it tries to show an interactive progress bar while touching, by printing "\r" to go back to the beginning of the line and overwriting with a new progress bar. I believe it does this at the beginning and the end of processing each file, which is why you're seeing 2 even though you're not touching (just displaying).
I agree this looks like a bug. I'll think about the best way to solve this and update this ticket when I do.
In the meantime, you can eliminate the excess lines with the following:
cat vmtouch.out | perl -pe 's/^.*\r//'
Thanks for the report!

I've encountered the same issue, reproduced with:
$ vmtouch -v /usr/bin/ls | cat -v
/usr/bin/ls
^M[OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO] 33/33^M[OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO] 33/33

           Files: 1
     Directories: 0
  Resident Pages: 33/33  132K/132K  100%
         Elapsed: 0.000152 seconds

My use case is to regularly display vmtouch output using the watch utility.
$ watch -n 1 vmtouch -v growing_file.bin

Having output repeated twice is undesirable.  Perhaps vmtouch could have a flag to disable the interactive progress bar and its use of \r.

Having looked at the code, the carriage return characters can be eliminated for the case where -v is used on the command line without -t.  The changes are available in pull request #97.


Keep track of bans/quiets

I thinnk it would be helpful to have ella see when there's a ban, and follow up with the op either in pm or #web-social for a reason and ban time. After the specified time, ella will ping the op to remove the ban. (I think we had a discussion about having the bot remove bans automatically, and we decided it wasn't a good idea, but that would also be an option if people are open to it)

I can create a bans table in the hashweb database, then have the bot POST to hashweb when someone is banned to update it.
We can then view bans and times in the browser (and cross reference them with the logs we already have)
We can also have the bot POST notes alongside each ban so we know why they're banned etc

Marking this as done
http://hashweb.org/stats/bans/


refactor: move some business logic out of the db class

Hello, I was wondering if you'd be willing to accept this PR. I thought the generation of the Token namedtuple and the AuthorizationCode should not be part of the db interface, when I started with it I was a bit confused.
Now for a user, implementing the interface of BaseDB is just about the database.
My current implementation isolates the "save" part, a bit similar to rest_framework mixins
I was also thinking about exposing the in-memory db as part of the library instead of the test, this would also help newcomers to test the library out.
Let me know what you think.
Regards!

Codecov Report

Merging #27 (2aae515) into master (188699e) will decrease coverage by 0.34%.
The diff coverage is 80.00%.

@@            Coverage Diff             @@
##           master      #27      +/-   ##
==========================================
- Coverage   99.27%   98.92%   -0.35%
==========================================
  Files          15       15
  Lines         549      557       +8
  Branches       56       56
==========================================
+ Hits          545      551       +6
- Misses          3        5       +2
  Partials        1        1

Impacted Files
Coverage Δ

src/aioauth/base/database.py
93.54% <80.00%> (-6.46%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 188699e...2aae515. Read the comment docs.

@synchronizing could you also review? I would like to merge. I think this might change the documentation on your side.

Looks good to me, and I think the changes are valid + a good step to better organizing the project. I'll make the correct adjustments on #21 to reflect these new changes when/if approved.
@Woile @aliev and I have spoken about a possible restructure of the project in the coming weeks/months. If you have any other suggestions that would be lovely. 👍

I have some ideas, but mostly thoughts, I'm still studying the library, so sorry if they don't make sense 😅  haha
I was thinking maybe providing some store implementations + an in-memory implementation (to play with, at the moment the in-memory is part of the tests), kind of like fastapi-users does [1] [2]
Also I was wondering if this should be aio-auth or asgi-auth, being asgi means it should work with starlette, fastapi and any other asgi library (kind of like wsgi with flask, django, etc). But this is mostly a thought rather than an idea, not sure the impact, though asgi servers provide the request object [3].
But don't hold up to that thought, maybe is also better to provide interfaces for different frameworks, like it is documented now in the README. I still have to weight in the different approaches, and if an asgi approach is even possible (I think it would work for middlewares... but providing the endpoints may be super difficult, so maybe just to have each framework implementation isolated.
Maybe the user should do: pip install aioauth[sqlalchemy,starlette] ???
For the docs I'd use mkdocs.
I'd also recommend using the pyproject.toml format, or separating the requirements into normal + development requirements (wheel and twine are not "test", so semantically is a bit confusing).
I maintain commitizen, so if you were to use conventionalcommits + semver, you could get automatic releases + changelog [4] [5]
Thanks a lot for the package and I hope I can contribute more.
If I come up with any real useful idea I'll open a ticket and we can discuss there 👍🏻


py3.5 install issue

got a pip issues on conda, py3.5, win10x64
>pip install pex
Collecting pex
  Downloading pex-1.1.14-py2.py3-none-any.whl (105kB)
    100% |################################| 112kB 578kB/s
Collecting setuptools<20.11,>=2.2 (from pex)
  Downloading setuptools-20.10.1-py2.py3-none-any.whl (509kB)
    100% |################################| 512kB 967kB/s
Installing collected packages: setuptools, pex
  Found existing installation: setuptools 27.2.0
Cannot remove entries from nonexistent file c:\miniconda3\envs\conda-rubito\lib\site-packages\easy-install.pth

(yay issue #300(now recursive...kinda))

this looks like a pip issue rather than a pex issue. it appears to be deinstalling your existing setuptools in favor of a lower version required by pex. the latest version of pex should be more permissive in its requirement range for setuptools, which I believe will inherently solve this problem for you.


Package functions added and errors corrected

Coverage increased (+0.1%) to 50.99% when pulling 234ce605a808c533e23067a999c46ff4750bf1da on compae:bugfix/deb-rpm-creation into 1bd772155eff74eec2b2845d084e1cf462f76bad on Stratio:master.

Coverage increased (+0.07%) to 50.912% when pulling 6978dc5ccf4805d6611492dd3f0f7ff183b6b198 on compae:bugfix/deb-rpm-creation into 1bd772155eff74eec2b2845d084e1cf462f76bad on Stratio:master.


Centralized Middleware

This pull request adds a Centralized Middleware which allows a centralized operating mode.
In this mode, the Agent is in charge of managing the Topics that Clients subscribe or publish.
It involves a significant reduction in memory footprint because of the absence of Fast RTPS.
This pull request has involved the following changes:

The Middleware read policy has been changed by a more "C-style" file one.
A CedMiddleware class has been added.
A set of CedEntitites have been added.
The FastMiddleware has been moved in another folder.
Several CedMiddleware tests have been introduced.

Build status:

Linux
Windows

Build status:

Linux
Windows

Build status:

Linux
Windows

Build status:

Linux
Windows

Build status:

Linux
Windows

Build status:

Linux
Windows


[#85] Update README

Closes https://github.com/hadshirt/sorting-algorithms/issues/85

table is broken in the pr


Datadog documentation fix

2 properties marked as deprecated here https://docs.datadoghq.com/agent/docker/?tab=standard for datadog chart

@SychevIgor Thanks for the contribution. Any time there is a change to the chart, even a documentation change, the version property needs to be updated. Can you please increment the version in the Chart.yaml file.


Items proxy api

Fixes #6475
Also makes a request to the proxy api and updates the items table with the status of each item

This will be behind a toggle - can we add an e2e test that turns that toggle on?

Yep, we can do that - I'll create another issue for adding the e2e test

@jamieparkinson @gestchild the routing is handled here
I suggested using catalogue but we might want to use works as that's what we're called it.


Parallelism

I have your /examples/mnist.py file working on an AWS GPU server. I needed to delete the "time_limit=12 * 60 * 60" part though because that was giving an error. My question is, does it use multiple cores by default? I read the Parallelism section of your paper, but am unclear if I have to do anything to make it parallel. What if for example I want to run it on a CPU-only server with 128 cores, would it automatically use all of them? Also, when you say it runs multi-GPU, do you mean within the same machine, or can it also run over a cluster of multiple GPU servers?
Also, in Table 1 of your paper, where you compare Auto-Keras with similar programs, are those accuracy results on the test set or validation set? In your github repo it seems to be only split into training and test sets. Should that also include a validation set?

Ignore my question about the accuracy results. I see now it is scored on the test set. So my only question is about parallelism.

I tried it on a CPU server now with 40 cores, and it seemed to be using all of them automatically.


INCLUDE DependsOn with class packages in different namespace

In my case, I have my main in package x.y
and want to INCLUDE / DependsOn a file in package x.a as well as in package x.y.z
@file:DependOn(...)

@file:Include("../helpers/CamelCase.kt")
@file:Include("./mappings/Jira.kt")

@file:EntryPoint("com.hoffi.something.AppKt")

package com.hoffi.something

import DependsOn
import EntryPoint
import Include
import com.hoffi.something.mappings.Histories

fun main(args: Array<String>) {
    App().main(args)
}

class App : CliktCommand() {
    override fun run() {
        ...
    }
}

doesn't find the class which is one of the defined classes in ./mappings/Jira.kt
package com.hoffi.something.mappings

import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import java.time.Instant

...
@SerialName("histories")
@Serializable data class Histories(
    val author: Author,
    @Serializable(InstantSerializer::class) val created: Instant,
    val id: String,
    val items: MutableList<Items>,
)
...

gives me:
/tmp/tmp17398417963705588090.tmp/scriptlet.248cdc652452029b.kt:18:18: error: unresolved reference: something
import com.hoffi.something.mappings.Histories
                 ^

btw: ... it took me a long time to really understand your comment you added in https://github.com/holgerbrandl/kscript/commit/ee7748e8843e474ad41535a3d9d33852ac56fb02

I guess this use case is not supported, and it's unlikely that this will ever work. @file:Include works by literally rewriting your script code prior to compilation to include the include-file at the given location. kscript tries to be a bit more clever, to at least sort imports to the top of the merged script. But it can't resolve the conflict you've created in your example where a single source file has 2 package statements. This is not legit according to the kotlin language specification.
Does this answer help?


test(linter/no-useless-spread): ensure spreads on identifiers pass

Closes #5405

#5561  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @DonIsaac and the rest of your teammates on  Graphite

Adds trivial, already-passing cases, so I'm just gonna merge this.


Add ability to mock kernel feature prober and expand BPF map tests

See commit msgs.
Followup from https://github.com/cilium/cilium/pull/14853

test-me-please

test-me-please

test-me-please


Add GitHub Action for code testing

This will allow for verification of incoming pull requests and promote testing code
Relevant actions:
https://github.com/marketplace/actions/rust-cargo
https://github.com/marketplace/actions/rust-toolchain

I can add this right after merging https://github.com/barter-rs/barter-rs/issues/5 on Friday

Closing this now we've migrated this repo to the new barter-rs monorepo:
https://github.com/barter-rs/barter-rs


Request to implement [ShouldSerialize] as attribute instead of adding an extra method

I want to control when to serialize and when to deserialize where [JsonIgnore] and [JsonProperty] won't cut it so I found out about ShouldSerialize
It's a shame that it's not a simple and straightforward attribute:
public class Employee
{
    public string Name { get; set; }
    public Employee Manager { get; set; }
    public bool ShouldSerializeManager()
    {
        // don't serialize the Manager property if an employee is their own manager
        return (Manager != this);
    }
}

vs
public class Employee
{
    public string Name { get; set; }
    [ShouldSerialize]
    public Employee Manager { get; set; }
}

Reference: http://www.newtonsoft.com/json/help/html/conditionalproperties.htm

[ShouldSerialize]
public Employee Manager { get; set; }

How it should perform a check in this case? You have to have a logic described somewhere.
It would be either having a separate attribute class for each property, or passing it a method/type describing the serialization logic (which is not much different from the current approach).
Or is your concern about having this method as public on a contract class? Then create a custom contract resolver and provide your own ShouldSerialize implementation.

@pieceofsummer @JamesNK Could a lambda be used in the attribute?

It would be simpler if ShouldSerializeMemberName could be a property instead of a method

@zmp2000 What about arguments?

@zmp2000 What about arguments?
I think there are no arguments passed on actual version.

What I'm aiming is:
Supose you have this class
public class Car{
public string color{get;set;}
}

And In a Page A you want to show the Color of your car to your user
but in a Page B you don't want to show
So you change your class to
public class Car{
public string color{get;set;}
public bool showColor {get;set;}
  public ShouldSerializeMemberName(){
   return showColor;
  }
}

and on your PageB method you set the variable showColor to false
So it works, now suppose instead of 1 property you have 20 properties that you want to Hide
So you Write 20 new properties and 20 new methods
My point is, we could create just 20 properties and set them directly instead having a method for each property reading another property to make it visible or not!
And a property allows you to have a logic if you want it to show
like
public class Car{
  public string Color{get;set;}
  public bool ShouldSerializeMemberName{ get {return Color=="Blue"}}
}


6_Design

Using our backlog and the original site as a guide, create a Wireframe of the site our group will build.

[ ]  Discuss must-have user stories in our backlog
[ ]  Create a wireframe to describe what the website will look like without any detail, just big blocks and fake text

Technology:
https://excalidraw.com/

@EliasMlopez99 hey I just want to check if you have time to review the changes that we discussed 😊 ?

Yes of course, i am on it


Allow passing next(error, job) to both cancel and update

When processing a job, I sometimes want to both update the job and cancel it - such as setting a code indicating the reason for cancellation. The code for this tends to look like this:
job.cancelReson = Cancellation.USER;
job.update();
const error = new JobError('canceled');
error.cancelJob = true;
next(error);

However, presumably the 'next' method is going to mutate the job record. It would be convenient to be able to do both in a single database update.
My proposal would be that passing both parameters (error and job) to next() would accomplish this.

That's a simple one. I don't see why we couldn't do that. It would be a point release also.

So it turns out this is getting more complex than I realized. I've got the following code pattern about a dozen places in my code:
job.someProperty = foo;
job.update();
job.addLog({}, 'some log message');
next(Error('error'));
So this is hitting the database three times. And what I haven't shown is that we actually need to wait on the first two promises - both the property update and the log addition.
(An example of where this would happen is if I get a job failed event from the cluster, I want to record what actually happened.)
A similar pattern also appears about a dozen times, which is that we're doing the exact same thing except outside of the context of a Queue.process call. In this case, the code looks like this:
job.someProperty = foo;
job.update();
job.addLog({}, 'some log message');
Queue.cancelJob(job);
Again, hitting the database three times. (In fact, I'm hitting it four times, because if it's a sub-task change I also wake up the parent task).
I wonder if there's a way to make a bunch of changes to a job as a single transaction?

Here's an alternative idea, which is a bit more radical:
Queue.process((job, control) => {
  control
    .update({ /* properties */ }) // set some properties
    .log.info('info message') // Log an info message
    .reschedule(1000); // Accepts either a date or a numeric delta
});

Queue.process((job, control) => {
  control
    .setStatus('failed')
    .log.error('error message')
    .cancel(); // You have to call reschedule, cancel, or fail as the last method.
});

Queue.getJobControl(id).then(control => {
  // Same interface but outside of process loop.
});

I'm not keen. This this would be mostly to support your rather complex requirements.
I follow the KISS principle wherever possible.
You can drop one extra db connection though @viridia.
Rather than using the Job.addLog call which does connect and update the database, just push a new log onto the Job.log array.
If you look at this code you will see the log objects are just simple objects. Make one in your code and then do a Job.log.push(newLogObject) before you call Job.update().
I'm only keen to make the titled suggestion so far.

I've decided to go in a different direction, so I'm happy to table this discussion.

Still with this queue? I wouldn't be surprised if you came up with a better solution.

Well, as you pointed out, my needs are complex. And I didn't even mention some of the other issues.
For example: users are going to want to monitor the status of their rendering jobs in real time, which means publish/subscribe to the browser. I'm using deepstream.io for this, but one side effect is that an event published to a topic is visible by all subscribers, which means that you don't want two worker processes to publish the same event (because everyone would then get double updates).
That in turn means that worker processes have to be careful about what events they listen to and what messages they broadcast. So listening to Node events on the queue is out, since every worker is going to get a copy of the same event from rethinkdb. Instead, I have the individual workers push updates to the appropriate subscription topic at the point of mutation.
Also, for my case it makes sense for logs to be in a separate table, since I want to have a separate pub/sub stream for the logs so that users can watch them in real time.\
Plus I'm having fun playing around with async / await :)

So are you going to roll your own queue?
It took me six months to write rethinkdb-job-queue (not full time). Trying to duplicate that will be a big job however in your case it could be better.


Laravel 10.x Shift

This pull request includes the changes for upgrading to Laravel 10.x. Feel free to commit any additional changes to the shift-80659 branch.
Before merging, you need to:

Checkout the shift-80659 branch
Review all pull request comments for additional changes
Run composer update (if the scripts fail, add --no-scripts)
Clear any config, route, or view cache
Thoroughly test your application (no tests?, no CI?)

If you need help with your upgrade, check out the Human Shifts. You may also join the Shifty Coders Slack workspace to level-up your Laravel skills.

:x: Shift was unable to find some of your controllers and left them as string references in your routes.
You will need to search for the following controller names in your routes and replace them with the appropriate static ::class reference manually.

[ ] authenticatedUser
[ ] destroyUser
[ ] index
[ ] login
[ ] logout
[ ] read
[ ] readAll
[ ] register
[ ] remove
[ ] removeAll
[ ] show
[ ] storeAvatar
[ ] updatePassword
[ ] updateProfile

:x: Shift could not upgrade the following files since they differed from the default Laravel version. You will need to compare these files against the default Laravel 10 versions and merge any changes:

[ ] app/Http/Middleware/Authenticate.php

:warning: Shift upgraded your configuration files by defaulting them and merging your true customizations. These include values which are not changeable through core ENV variables.
You should review this commit for additional customizations or opportunities to use new ENV variables. If you have a lot of customizations, you may undo this commit with git revert 6127cab2 and make the config file changes manually.

:information_source: Shift updated your dependencies for Laravel 10. While many of the popular packages are reviewed, you may have to update additional packages in order for your application to be compatible with Laravel 10. Watch dealing with dependencies for tips on handling any Composer issues.

:information_source: Laravel 10 added PHP type hints to all user-land code included in a new Laravel application. In an effort to modernize your code, Shift added type hints to any method which is used by Laravel, as well as any method which had types defined in its PHP DocBlock.

:information_source: Now with type hints in your code, defining types within PHP DocBlocks is redundant. Laravel has removed all of the @param and @return tags from its DocBlocks where types are defined with PHP. Similarly, Shift removed these tags from any DocBlock where the code now has equivalent type hints.

:information_source: Shift understand developers have different preferences when it comes to type hints. All of Shift's automation is done in nice, atomic commits. This makes it easier to undo any of the changes Shift makes.
If you wish to undo any of these changes, you may first revert the DocBlock changes by running git revert 3ef105e. Then undo the added type hints by running git revert 9f0f050.

:information_source: All of the underlying Symfony components used by Laravel have been upgraded to Symfony 6.2. Shift detected references to Symfony classes within your application. You should review the Symfony change log for any additional changes.

:information_source: Laravel began using Vite to build frontend assets in Laravel 9.19. While you may continue to use Laravel Mix, it is no longer the default. If you wish to modernize your application to use Vite, you may run the Vite Converter for free.

:warning: Laravel 10 now requires PHP 8.1 or higher. You should verify the PHP version in your environments to ensure it meets this new requirement.

:tada: Congratulations, you're now running the latest version of Laravel!
Next, you may optionally run the following Shifts to ensure your application is fully upgraded, adopts the latest Laravel conventions, and easier to maintain in the future:

Upgrade Checker ensures your application is fully upgraded by detecting any outdated code.
Laravel Fixer automatically updates your code to the latest Laravel conventions.
Tests Generator intelligently generates model factories, HTTP Tests, and configuration for your application.
CI Generator intelligently generates CI jobs to lint PHP, check code style, and run tests, including Dusk.

You may also use the Shift Workbench to automate common tasks for maintaining your Laravel and PHP applications.

:alembic: This Shift is still being refined. Please report any issues or suggestions to<EMAIL_ADDRESS>Your feedback is what helps improve the experience for everyone.


Portainer/Portainer-CE:latest has lost ability to create containers\stacks since last image update

Bug description
Login page states 'Unauthorised' and "Unable to locate environments' before login has been attempted
When logged in cannot create containers or stacks because the buttons are not clickable (font is light grey, not white)
Expected behavior
The login page should not be saying unauthorised before login is attempted.
Should not have any issue finding the environment that has always been there.
Administrator level accounts should have full functionality
Buttons to execution actions should be available
Portainer Logs
time="2022-07-23T11:59:41Z" level=info msg="Encryption key file portainer not present"
time="2022-07-23T11:59:41Z" level=info msg="Proceeding without encryption key"
time="2022-07-23T11:59:41Z" level=info msg="Loading PortainerDB: portainer.db"
time="2022-07-23T11:59:42Z" level=info msg="Opened existing store" version=51
2022/07/23 11:59:42 server: Reverse tunnelling enabled
2022/07/23 11:59:42 server: Fingerprint 08:e4:44:34:b6:00:a2:08:bd:6a:1a:be:6d:80:65:a3
2022/07/23 11:59:42 server: Listening on <IP_ADDRESS>:8000...
time="2022-07-23T11:59:42Z" level=info msg="[INFO] [cmd,main] Starting Portainer version 2.14.1\n"
time="2022-07-23T11:59:42Z" level=info msg="2022/07/23 11:59:42 [INFO] [http,server] [message: starting HTTPS server on port :9443]"
time="2022-07-23T11:59:42Z" level=info msg="2022/07/23 11:59:42 [DEBUG] [chisel, monitoring] [check_interval_seconds: 10.000000] [message: starting tunnel management process]"
time="2022-07-23T11:59:42Z" level=info msg="2022/07/23 11:59:42 [DEBUG] [internal,init] [message: start initialization monitor ]"
time="2022-07-23T11:59:42Z" level=info msg="2022/07/23 11:59:42 [INFO] [http,server] [message: starting HTTP server on port :9000]"
time="2022-07-23T11:59:50Z" level=info msg="2022/07/23 11:59:50 http error: A valid authorisation token is missing (err=Unauthorized) (code=401)"
time="2022-07-23T11:59:50Z" level=info msg="2022/07/23 11:59:50 http error: A valid authorisation token is missing (err=Unauthorized) (code=401)"
time="2022-07-23T11:59:50Z" level=info msg="2022/07/23 11:59:50 http error: A valid authorisation token is missing (err=Unauthorized) (code=401)"
Steps to reproduce the issue:

Go to 'login page'

See errors in top right of screen

Try to create container or stack

Fill out all information or load customer template\yaml editor

Try to click button to execute and create container\stack

Technical details:

Portainer version: 2.14.1
Docker version (managed by Portainer): 20.10.3-1306
Platform (windows/linux): DSM 7.1-42661 Update 3
Command used to start Portainer: /portainer
Browser: Microsoft Edge
Use Case (delete as appropriate): Using Portainer at Home, natively in Docker application on Synology
Have you reviewed our technical documentation and knowledge base? Yes

Additional context
The issue first began 12 days ago when the Portainer version increased.
No changes were made to any settings including execution scripts and variables which are default from when first created a year ago
Image is Portainer/Portainer-CE:latest

If you roll back to the previous version, does the issue go away?

Having the same issues on WD MyCloud EX4100. Button to deploy the stack is not clickable

I have the same issue on 2.13 as well, I'd guess there's an error somewhere but the message isn't being shown?

I hadn't updated my portainer in a while and had to downgrade back to 2.11.1 because no environments were listed

Sorry for coming back to this so late everybody. I found a work around a while ago. In a Chromium based web browser right clicked on the disabled Create Custom Template button and select Inspect from the context menu. You will be taken straight to the Dev tool for the web browser, the element will shown and you need to remove the 'Disabled=Disabled' attribute at the end of it. No refresh of the browser will be necessary, the button will active.


Limit sequence numbers between ranges

Changes
Set ranges for sequence number wraparounds based on previous logic in sdx-collect.
How to test
Request each of the endpoints /sequence, /batch-sequence, /image-sequence and check they start at the appropriate values.
Who can test
Anyone but @iwootten

Looks good 👍


Need DSL for format and type data

Currently, users need to author a ps1xml to update format and type data.  We should expose a DSL to make this simpler and more aligned with PowerShell script than using xml.  This needs to be an RFC.

Will resolved by #7857 PSMore.

This needs to be an RFC.

I suspect in the intervening years since this issue was opened, we've come to prefer discussions in issues over RFCs. I should also note that our need for a DSL isn't as great as it used to be, and after going down the path of trying to implement such a DSL we've learned a few things along the way about the complexity of setting up a framework for one.

Dunno about everyone else, but I'd certainly be happy with having a couple of cmdlets to handle this instead of just a full-blown DSL. Even just making the current C# APIs easily accessible from PowerShell and allowing the resulting format definitions to be registered would be a vast improvement from what we have currently.

PowerShell benefits from balancing dynamic typing and strong typing. I guess PowerShell classes is best for strong typing and it makes you look again in the direction of PSMore idea. Also it says that it is better to invest in PowerShell classes.

but I'd certainly be happy with having a couple of cmdlets to handle this instead of just a full-blown DSL

Those are my thoughts too. A DSL would be nice, but I'd see that as being implemented in some set of cmdlets.
I'd certainly like to get rid of the XML, and given the way we embed scripts in it currently, some script-based solution seems appropriate.

Also it says that it is better to invest in PowerShell classes

While I agree that classes should be well suited to this, I think we've got more work to do on classes first. OTOH, I think this would be a good motivating factor to fix classes.

I'd certainly like to get rid of the XML, and given the way we embed scripts in it currently, some script-based solution seems appropriate.

It seems @daxian-dbw have a tool to convert the psxml-s to internals. Perhaps it could be enhanced to cmdlet.


[opencc] update to 1.1.9

Usage failed with the following error:
1> [CMake] CMake Error at F:/PRreview/vcpkg/installed/x64-windows/share/opencc/OpenCCTargets.cmake:56 (set_target_properties):
1> [CMake]   The link interface of target "OpenCC::OpenCC" contains:
1> [CMake]
1> [CMake]     PkgConfig::marisa
1> [CMake]
1> [CMake]   but the target was not found.  Possible reasons include:
1> [CMake]
1> [CMake]     * There is a typo in the target name.
1> [CMake]     * A find_package call is missing for an IMPORTED target.
1> [CMake]     * An ALIAS target is missing.

should be ok now, @LilyWangLL

/azp run

Rerunning due to network mistake last night.

Usage and feature tools test passed on x64-windows.


[HAL] Remove incorrect buffer_view.buffer folding

It is unsafe to fold buffer_view.buffer of buffer_view.create because create operation can create a buffer subspan, and to do the correct folding we have to know the buffer size.

safe :)


Update the valid hosts and TLDs based on the IANA list

👋 I was setting up a new .dev domain and noticed an issue getting it linked with hotline webring. This PR addresses that issue, and improves the accuracy of the host validator according to IANA's list of reserved domains.
This change addresses a few issues with the host validator:

.dev is now (as of 2019) a valid TLD (https://domains.google/tld/dev/)
IANA has a list of special use domain names which includes additional
reserved hosts and TLDs, some of which are for development
(https://www.iana.org/assignments/special-use-domain-names/special-use-domain-names.xhtml)

In this change, we update the host validator to:

Allow the .dev TLD
Disallow example hostname
Disallow the .local, .example, and .test reserved TLDs
Update all example.com references in specs to notareal.domain
since example.com no longer validates

Let me know your thoughts on this!

Looks like CircleCI has the "build PRs from forks" setting disabled. I'm going to turn on that setting for future PRs and just merge this one as-is.


[UI] Running actions in forked repo will cause status dupe

Description
The UI goes wrong when click as the following steps:

Actions -> (Any) Workflow Job
Click (Any) Tab

Affected positions:

How to replicate the issue
(Basically any forked repo with actions enabled can do this, for example https://github.com/01101sam/node-https-proxy-agent)
https://user-images.githubusercontent.com/100821827/178733445-d810ae82-9615-4e7f-8b0f-d4fcd9f4b285.mp4
Extension version
22.7.11
Browser(s) used
Chrome

Duplicate of https://github.com/refined-github/refined-github/issues/5755


BUG: use 11.0 as minimum macOS platform ABI tag on arm64

Fixes #524.

The test failure is odd. I would have thought that packaging would refuse to generate a tag on macOS with an ABI version lower than 11.0 on arm64. I'll see if there is a more suitable packaging function to use in this test.

This highlights that we have a test coverage gap that we should fix (can be done later, I can test locally for now).

It mostly highlights that my laptop is still an Intel mac. We don't test on architectures other than amd64 for any other platform. Extending the CI to run on other architectures would be nice, but not very high priority.

We're under the mesonbuild org, so it's not possible to see our credits usage on Cirrus CI - and hence I'm not sure if we can add arm64 on Cirrus. Meson itself doesn't use Cirrus though, so it seems like we can try without causing an issue; worst-case we'd exhaust free credits in one month.

AFAIK Cirrus CI does not use a credit system for using their service within the limits allocated to public repositories. https://cirrus-ci.org/faq/#are-there-any-limits So far we don't use any macOS worker on Cirrus CI so far, so we can setup one to run on arm64 and still have all the workers run in parallel

Correction. The page I linked to actually reports that there is also a monthly usage cap. I don't think we are likely to hit the 10000 CPU minutes limit for Linux tasks, but the 500 minutes per month for macOS task is not very high. I'll setup the macOS arm64 jobs and see how we do.

There does not seem to be a better packaging API to use for the test. I fixed it the ugly way. The macOS arm64 CI job seems to be running fine.


RSTN data access and loading capabilities

Description
The Radio Solar Telescope Network (RSTN) is often used by the community, both the 1s fixed frequency data available for 8 frequencies (245, 410, 610 MHz and1.4, 2.7, 5, 8.8, 15.4 GHz) from 4 worldwide networks (Learmonth, San-Vito, Sagmore-Hill, and Paehua) that have different time coverages, and also the dynamic spectra data (25-180MHz) too.
It would be nice to be able to search for this data and then have readers for the text files for the fixed frequency data and for the binary spectral data files, and radiospectra is probably the best place for this.
The data is available here:

Fixed frequency: ftp://ftp.ngdc.noaa.gov/STP/space-weather/solar-data/solar-features/solar-radio/rstn-1-second/

Dyanmic spectral data: ftp://ftp.ngdc.noaa.gov/STP/space-weather/solar-data/solar-features/solar-radio/rstn-spectral/

Developing a client would be very straight forward, and I already have some readers that can be added here too.
thoughts?

Hey I totally missed this so this is kinda already done in #44 have added client and specific spectrograms for Callisto, EOVSA, RSTN, SWAVES, PSP review most welcome

Closed in #44


Breaking quasiquote pattern matching behavior change in 4.4.16

Not sure if the change is intended, but the following snippet works using 4.4.15-, but breaks with a MatchError using 4.4.16.
import scala.meta._

val q"..$_ class $tname[..$tparams] ..$_ (...$paramss) extends $_" = "@foo class Bar[T](val baz: Bazz[T, U]) {}".parse[Stat].get
println(tname)

Scastie:

Using 4.4.15 (prints: Bar)
Using 4.4.16 (breaks: scala.MatchError: @foo class Bar[T](val baz: Bazz[T, U]) {} (of class scala.meta.Defn$Class$DefnClassImpl))

Thanks for reporting! Reading this again I think it's due to the change I did recently. Template now includes the extends keyword
This seems to work:
import scala.meta._

val q"..$_ class $tname[..$tparams] ..$_ (...$paramss) $_" = "@foo class Bar[T](val baz: Bazz[T, U]) {}".parse[Stat].get
println(tname)

Just with removed extends - sorry I didn't realize this sooner.

@robby-phd if removing extends, as @tgodzik suggested, fixes the problem, the change is as intended.

Yes, that works: https://scastie.scala-lang.org/rIEv9CISRcOyR0hvQ4AABQ
I'm closing this issue since the change seems to be intended.


add git lens to git trick demo

Description
add my git trick
Checklist

[x] I made sure that the markdown files are formatted properly.
[x] I made sure that all hyperlinks are working.

@nils-imhoff Could you please check the failing linter and also rebase on main?


Zippos don't exist and should actually not be able to do this.

Fixes #10723

How would a lighter ever fit in a spot the size of 1 cigarette

well cigarette packets dont fit 6 cigs irl anyway so its not realistic either way
maybe theyre massive cigarettes

maybe theyre massive cigarettes

well cigarette packets dont fit 6 cigs irl anyway so its not realistic either way

But that's for gameplay reasons, in real life you hopefully last longer with a full packet than an SS14 round, but how long they take to smoke up is about similar between both.

Okay, eggs and pizza slices are apparently also size 1 so this is fine with me.


File manager - Add a close button to the context menu?

If i open the context menu it is not easy to close it again without selecting other files. To close the menu i need to do it on the exact same row i opened it on to avoid selecting other files. I can also close it by pressing on an empty space but with many files listed it will fit the whole page so there is no free space to click on.
So it would be nice if the context menu did have a close entry (or an X) that will close the menu.
I know i can use the ESC button but it should be possible to close it with the mouse too.

Hi,
You can easily close the contextmenu by clicking anywhere on the page, not necessarily clicking in the table. Besides escape is an awesome key, you should learn to love it. :wink:
I don't like an idea of having close button for context menu, it sounds bizarre. Sorry.


Lotus release pipeline re-architecture, validation, and observability

eta: 2023Q1
description:
Re-architect the build pipeline to support additional build targets and CD of new lotus releases to Web3 GitOps Platform. Validate releases after they are published, including running brew install, apt-get install, etc on the various platforms we target. Provide better developer insights by leveraging CircleCI slack notifications and metrics on released / running nodes to determine when things have failed and notify the team.

We are not pursuing this milestone anymore


Setup for the NumPy speed comparison?

On your website, you report that ND4J runs twice as fast as NumPy for matrix multiplication.
This strikes me as highly questionable. In both cases, NumPy and ND4J are doing matrix multiplication by calling out to an external BLAS. So you're really measuring the speed of the external BLAS. Moreover, ND4J needs to copy data as it is goes in and out of the JVM, which is often not the case for NumPy.
My guess is that you're linking to a different BLAS with NumPy, which results in the observed speed difference.

We'll be adding to this here soon with different versions of blas. Please come on gitter if you'd like to ask more questions about this. Thanks!

Hi, Just run these codes on the same computer, turns out Numpy is about 5 times faster than ND4J, correct me if I am wrong.

import java.sql.Timestamp;
import java.util.Random;

import org.nd4j.linalg.api.ndarray.INDArray;
import org.nd4j.linalg.factory.Nd4j;

public class Main {

	public static void main(String [] args) {
		Random rand = new Random();
		int length = 1_000_000;
		double[] d1 = new double[length];
		double[] d2 = new double[length];

		for(int i = 0; i < length; i++) {
			d1[i] = rand.nextDouble();
			d2[i] = rand.nextDouble();
		}

		INDArray nd1 = Nd4j.create(d1, new int[] {1, length});
		INDArray nd2 = Nd4j.create(d2, new int[] {length, 1});

		long t1 = timeNow();

		INDArray nd3 = nd1.mmul(nd2);

		long t2 = timeNow();
		print(nd3);

		double d = 0;
		for(int i = 0; i < length; i++) {
			d += d1[i] * d2[i];
		}

		long t3 = timeNow();
		print(d);
		print("ND4J cost time: " + Long.toString(t2 - t1));
		print("For Loop cost time: " + Long.toString(t3 - t2));
	}

	private static void print(Object o) {
		System.out.println(o);
	}

	private static long timeNow() {
		return System.currentTimeMillis();
	}
}

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
  <groupId>group.id</groupId>
  <artifactId>nd4j-test</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>nd4j-test</name>
  <description>nd4j-test</description>
  <properties>
  	<maven.compiler.source>1.7</maven.compiler.source>
  	<maven.compiler.target>1.7</maven.compiler.target>
        <exec.mainClass>Main</exec.mainClass>
  </properties>
  <dependencies>
  	<dependency>
  		<groupId>org.nd4j</groupId>
  		<artifactId>nd4j-native-platform</artifactId>
  		<version>0.9.1</version>
  	</dependency>
  	<dependency>
	    <groupId>org.slf4j</groupId>
	    <artifactId>slf4j-log4j12</artifactId>
	    <version>1.7.25</version>
	    <scope>test</scope>
	</dependency>

  </dependencies>
</project>

Output is:
250,561.16
250561.20823672338
ND4J cost time: 5
For Loop cost time: 274

Meantime, the ND4J codes below:
import numpy as np
import time

length = 1000000

a = np.random.rand(length)
b = np.random.rand(length)

t1 = time.time()
c = np.dot(a, b)
t2 = time.time()

print(str(1000 * (t2 - t1)))

The output is:
0.638008117676

Sorry for not carefully handle the test, after test them with 100 iterations:
double all = 0d;
                for(int i = 0; i < 100; i++) {
                        long t1 = timeNow();
                        nd1.mmul(nd2);
                        long t2 = timeNow();
                        all += (t2 - t1);
                }

                print(all / 100);

Output:
98.3

all = 0.0
count = 0
for i in range(100):
  t1 = time.time()

  c = np.dot(a, b)

  t2 = time.time()
  count += 1
  all += (t2 - t1) * 1000

print(str(all / count))

Output:
221.279616356

Yes, your are right, ND4J is almost twice faster than Numpy

@cinqs Do you have a reason why ND4j should be faster? I think the argument of @shoyer sounds reasonable:

In both cases, NumPy and ND4J are doing matrix multiplication by calling out to an external BLAS. So you're really measuring the speed of the external BLAS.

@cinqs Do you have a reason why ND4j should be faster? I think the argument of @shoyer sounds reasonable:

In both cases, NumPy and ND4J are doing matrix multiplication by calling out to an external BLAS. So you're really measuring the speed of the external BLAS.

The differenc eis probably due to loop unrolling in the java code. Can make a better comparison - strictly between numpy and ND4J that is - by disabling that optimization with XX:LoopUnrollLimit.


If an Account External ID is selected in Data Import Settings, that should control the Account Matching behavior rather than name alone

Hub thread: https://powerofus.force.com/0D58000002MwUzK

**lurch: add

Tracking W-008638

Included in beta release version 3.65 (Beta 1) and higher

Included in production release version 3.65


Issue with n_job = -1

Hi! I'm a student of Computer Engineering and I'm trying to use SOMPY. I want to speed up the algorithm (reducing the total time elapsed) and I see that in the code the train function has a parameter (njob) that can help me. I have been studying this function and supposedly I have to put njob = -1 in order to use all the cores of my processor. The problem is when I call the train function with njob = -1, the algorithm freezes and it does nothing. The only way that works fine is with njob = 1, because if I put njob = 2 or njob = 6, it finishes later than with njob = 1, so I can't understand (with njob = 6 it supposed to run faster that with njob = 1).
I REALLY NEED a working paralelization of this algorithm in order to run faster for large maps (100x100 approximately), so if there is a solution to this problem I will be really grateful if you can help me.
Thank you for your time.

SOMPY uses "Parallel()" function from sklearn library to find the BMUs by chunks. As far as I remember, this function pickles the data, i.e. the chunks, for sharing it in jobs... If the chunks are too small, the pickle step may make the process slower than non parallelising. This is my hypothesis of what's failing here.
I would try to use the multiprocessing library to accomplish that. If I have time I will try it to see if I can make it work!

I just wrote a (hopefully =D) solution in PR #47


Add super call to constructors in transport

Signed-off-by: John Mazanec<EMAIL_ADDRESS>Description
Small fix: add super call in constructors in transport folder. This is for consistency's sake.
Check List

[X] Commits are signed as per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Codecov Report

Merging #169 (d94df57) into main (549b8ac) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##               main     #169   +/-   ##
=========================================
  Coverage     82.66%   82.66%
  Complexity      803      803
=========================================
  Files           117      117
  Lines          3582     3582
  Branches        338      338
=========================================
  Hits           2961     2961
  Misses          466      466
  Partials        155      155

Impacted Files
Coverage Δ

...ch/knn/plugin/transport/TrainingModelResponse.java
100.00% <100.00%> (ø)

...n/plugin/transport/UpdateModelMetadataRequest.java
92.30% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 549b8ac...d94df57. Read the comment docs.


[BEAM-13249][Playground] Security – Mock Network

Added mitmproxy to proxy all http/https outbound traffic and check that requested hosts are in the allow-list. Now allow-list contains only localhost.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[x] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

GitHub Actions Tests Status (on master branch)

See CI.md for more information about GitHub Actions CI.

R: @ilya-kozyrev @snkalinin

LGTM

R: @pabloem

this LGTM - but can we please detail in the documentation (wiki / readme / smething) what tools we use and how it works? thanks!

this LGTM - but can we please detail in the documentation (wiki / readme / smething) what tools we use and how it works? thanks!

@pabloem thanks for the review! We already documented it in the Security doc, feel free to take a look and leave comments if you think something else should be added.


Workspace Analytic Rule Template out of sync with GitHub

Describe the bug
The Analytics Rule templates in my Azure Sentinel workspace are not up to date with the templates on GitHub. A bug in a template was fixed on GitHub in 2020-Dec but the template in my workspace was last updated 2020-June. The bug causes malformed alerts to be generated, and since it is published my Microsoft I cannot fix it myself.
The only course of action is disabling the MS published templates since they are not reliable AND there is no policy stating how/when/if they will be updated.
To Reproduce
Steps to reproduce the behavior:

Go to any Azure Sentinel workspace
Click on 'Analytics'
Click on 'Rule Templates'
Search for 'Suspicious application consent for offline access'

ID = 6d4cfdf4-2b86-4f12-8a10-49151f23c990

Scroll down to `Template last updated' and see June 28, 2020
Compare to the GitHub template, which was last updated on Dec 9, 2020

Detections/AuditLogs/SuspiciousOAuthApp_OfflineAccess.yaml
the bug was fixed on Dec 2, 2020 in https://github.com/Azure/Azure-Sentinel/commit/315d3bfd1435b96dd30dd7713e78c2277ba762d0

Expected behavior
The template in my workspace to be kept reasonably up to date.
Screenshots

Additional context
This template is out of date in all of my Azure Sentinel workspace and it is not the only template that is out of date.
I raised a support ticket and they had the no helpful response other than maybe GitHub is not in sync with the MS Azure Sentinel tenant.
There are rigorous checks for detection syntax & structure...but absolutely no information about how those templates get distributed.

What is Microsoft's plan for pushing this updates from GitHub?
How will rules deployed from a template be updated when the template is updated?

Hi Michael,
I share your concern when it comes to the default rule templates. So I went back to when they announced how to use templates: https://techcommunity.microsoft.com/t5/itops-talk-blog/azure-sentinel-using-rule-templates/ba-p/2028427
It does suck that it doesn't get updated as frequently as github does, but the templates are only a start.
Have you considered updating and managing Sentinel manually using automation scripts? Such as described here: https://techcommunity.microsoft.com/t5/azure-sentinel/deploying-and-managing-azure-sentinel-as-code/ba-p/1131928
You could even update your own rules via powershell directly from github.

Hi @mpalumbo7 and @Lodewyk-Git : we seems to have add an issue with this template. The update should be deployed in a couple of days. We also work to identify the issue and improve our process.


SCM type VSTSRM not supported when decompling

Bicep version
Bicep CLI version 0.4.63 (7ebed03284)
Describe the bug
When running the decompile cmd from Azure CLI 2.24.2 I get an error from Bicep saying:
Warning BCP036: The property "scmType" expected a value of type "'BitbucketGit' | 'BitbucketHg' | 'CodePlexGit' | 'CodePlexHg' | 'Dropbox' | 'ExternalGit' | 'ExternalHg' | 'GitHub' | 'LocalGit' | 'None' | 'OneDrive' | 'Tfs' | 'VSO' | null" but the provided value is of type "'VSTSRM'".
This is the breviated resource:
{             "type": "Microsoft.Web/sites/config",             "apiVersion": "2018-11-01",             ...              "properties": {                 ...                 "scmType": "VSTSRM",                 ...             }  }
According to the documentation, VSTSRM is a valid input. Our repo is located in Azure DevOps.
https://docs.microsoft.com/en-us/azure/templates/microsoft.web/sites/config-web?tabs=json
To Reproduce
Setup an ARM template with a resource such as above.
Run az bicep decompile --file <template.json>

Can you share the original template? Can you try with a more recent api version? Looks like the latest is 2021-01-01


[resnet/unittest] Add unittest for resnet

This patch adds resnet18 unittest model generation with genModelTests.
Further, the input data range is changed from 0 to x from -x to x as
relu based models work better with 0 to x data range to prevent loss of
information.
This patch adds resnet models unittest for resnet18.
The verification has been done offline for 2 iterations for the output
of all layers with precision of 1.1e-4.
Derivaitves and gradients have higher error because of relu: when some
value is close to 0, it can be positive or negative with some error (of
the order of e-7). Although this error is way within the error limit.
however, this exacerbates the error in backwarding where derivatives (which
are significant in values) can flow if the relu value was over 0, and
not flow if under zero. This is manageable in smaller models but
difficult to avoid in unittests for larger models.

Other bug fixes in this patch:

max error reported by unittest_nntrainer_models has been fixed
error reporting now includes layer type as well
ModelTestOption MINIMUM has been renamed to NO_THROW_RUN

Resnet unittest is disabled as the golden data exceeds 70MB. This will be enabled when the golden data can be generated at runtime.
Signed-off-by: Parichay Kapoor<EMAIL_ADDRESS>

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #1578. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnstreamer.mooo.com/.

:octocat: cibot: @kparichay, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnstreamer.mooo.com/nntrainer/ci/repo-workers/pr-checker/1578-202109231315220.69390988349915-c29de3dcac3d6dd3d4f8b936bac7a2b365d3b31d/.


Versioning of CMIP6_CVs

I note that the releases of the CVs in this repository are explicitly linked to CMOR versions, would it be possible to (also) tag releases against the versions in the CMIP6 global attributes documentation, currently at 6.2.3?
Given that many of the changes over recent months are associated with adding information (e.g. institute_ids), rather than changing the structure of the CVs, perhaps an additional level of version numbering, i.e. 6.2.3.x, could be added and x incremented every so often (not necessarily after every change).

yes, please contribute to the discussion at https://github.com/WCRP-CMIP/CMIP6_CVs/issues/28
thanks,
Karl

@matthew-mizielinski this is a dupe of #28, so will close this and we can direct comments into the open issue


[Bug] Roads on web app on android not visible

Description
I have this bug for a long time, sometimes its on sometimes its not. When i load the map multiple times or even a few times a day, the roads dissapear and i only see a blank field and rivers, buildings and everything else except the roads and alleys, and refreshing or restarting the app doesnt work. I am using google maps javascript api because its a browser app so this bug only occurs (i think) on android, on my laptop (windows chrome) everything looks fine, so i assume its a cache problem but i already tryed reinstalling, clearing data and cache and it didnt resolve the problem but in development i dont have this bug on windows chrome. The curious thing is that tomorrow it will work fine for a bit, like every day it resets and works until it doesnt. Clearing cookies from the chrome settings for the specific site might fix it apparently, but its just temporarily
Steps to Reproduce
Loading the map multiple times might cause the bug
Environment

Library version: 1.1.0
Google maps version: weekly
Browser and Version: chrome 129.0.6668.54
OS: android 12
react 18.2.0

Logs
No response

Forgot to mention that this bug occured when i had 3D buildings enabled, now i am using three.js objects while i have the buildings off on the 3D vector map

Although I think it's unlikely this has anything to do with React and this library, I still love to have a look at that. If this only happens in combination with three.js / WebGlOverlayView, it might be related to some sort of problem with the GPU.

this morning the map wouldnt show the roads only the train tracks and pedestrian only streets marked with a black line

So i tryed clearing the cache, data, cookies, and restarting the app many times, created a new account, but with no luck this time
As seen in the image below im trying to render 3600 objects with three.js (const dummy = new THREE.Object3D();)

i remember that when i rendered 289 objects i didn't had this bug but now (with 3600) sometimes when i load the map it either crashes my app showing only a blank page or a default 2D map (im guessing a raster type)

In conclusion im going to try again to render max 300 objects and add/remove the objects based on the map bounds when user moves

That all sounds like the GPU crashes during rendering for some reason. What are you using to render the three.js objects?

const texture = new THREE.TextureLoader().load("/models/fog.jpg");
const buildGeometry = new THREE.BoxGeometry(1, 1, 1);
const buildMaterial = new THREE.MeshPhongMaterial({
map: texture,
depthWrite: true,
transparent: false,
opacity: 1,
});
buildMesh = new THREE.InstancedMesh(buildGeometry, buildMaterial, 4000);
scene.add(buildMesh);
const dummy = new THREE.Object3D();
// adding position, scale
dummy.updateMatrix();
buildMesh.setMatrixAt(i, dummy.matrix);
I have optimized the image and now has 727bytes instead of 6kb.

That all sounds like the GPU crashes during rendering for some reason. What are you using to render the three.js objects?

const texture = new THREE.TextureLoader().load("/models/fog.jpg");
const buildGeometry = new THREE.BoxGeometry(1, 1, 1);
const buildMaterial = new THREE.MeshPhongMaterial({
map: texture,
depthWrite: true,
transparent: false,
opacity: 1,
});
buildMesh = new THREE.InstancedMesh(buildGeometry, buildMaterial, 4000);
scene.add(buildMesh);
const dummy = new THREE.Object3D();
// adding position, scale
dummy.updateMatrix();
buildMesh.setMatrixAt(i, dummy.matrix);
renderer = new THREE.WebGLRenderer()
I have optimized the image and now has 727bytes instead of 6kb.

Same day yesterday I have updated the app showing ~267 objects but the map still wouldnt show the roads, even after clearing cookies, cache, data, reinstalling the pwa. The desktop version and the Edge browser on android was working fine.
Today the pwa on chrome works fine with the 267 objects, so I'm guessing its a cache thing, after consulting chatgpt it appears that the memory usage of the threejs objects can clear the cache from the memory RAM, where the google maps tiles are cached so the threejs scene can use that fresh memory to render its objects
"Conclusion
Yes, high memory usage from rendering a large number of Three.js objects can cause the browser to free up memory by clearing cached resources, which may affect the rendering of Google Maps tiles, potentially causing elements like roads and train tracks to disappear. To prevent this, it's important to optimize your Three.js scene to minimize memory usage and monitor performance using the browser's developer tools."
In the docs it says there are known issues with map-caching https://visgl.github.io/react-google-maps/docs/api-reference/components/map#map-instance-caching
So maybe it's not really a bug but maybe a method for optimizing the api to not be exploited by limiting the map-views.
This thread can now be closed/stay open or  moved somehwere else

All right, I will move this over to the discussions section then.
I saw you seem to be using your own THREE.WebGlRenderer so I'm curious, are you aware of the @googlemaps/three package? Or is the three.js stuff unrelated to the map and just happens to be on the same page?
If it is indeed a GPU crash of some sort, it should be fixed by either restarting the browser (i.e. force-kill the chrome app and relaunch) or by restarting the device (if it is a really bad crash). Everything else shouldn't affect this at all.
The disappearing roads would be worthy of a report as an issue in the google maps issue tracker here: https://issuetracker.google.com/issues?q=status:open componentid:188853&s=created_time:desc
To clarify the 'map instance caching': The only purpose of that is to prevent your app from creating a ton of map-instances when a single one can do the job (for example, you close the map in an SPA and come back to the map later). React would normally destroy the map instance and create a new one when needed, and our caching will keep the google.maps.Map object around in memory so it can be remounted instead of creating a new one. This isn't persisted anywhere beyond memory, so a reload will always clear that cache.


Deprecate Homebrew/homebrew-command-not-found

This doesn't seem to be used at all nowadays:
https://formulae.brew.sh/analytics/brew-command-run/365d/
As an official tap, it has overhead on e.g. RuboCop upgrades and a presence in Homebrew/brew CI, manpage, etc. that no longer seems to be warranted.
If there's enough 👍🏻 to do this: I'll add various odeprecated or odisabled calls to Homebrew/homebrew-command-not-found and archive the repository.

I don't think this would show on the command-run analytics, because it's not invoked directly.
Unless we're looking for the command-not-found-init command, which may not be used as opposed to reading the docs.
In saying that, I've personally never used it.

I don't think this would show on the command-run analytics, because it's not invoked directly.

I guess my thinking is: none of the relevant commands show up in the analytics.

In saying that, I've personally never used it.

This is an added datapoint: from a show of hands at the last Homebrew AGM iirc no maintainers used this.

This is an added datapoint: from a show of hands at the last Homebrew AGM iirc no maintainers used this.

I remember a few people raising their hands (including me...)

I remember a few people raising their hands (including me...)

@carlocab My bad, maybe it was homebrew-autoupdate people didn't use?
Any idea why there's no results showing up in analytics here? Would you be happy to take on any improvements to e.g. not committing data to repo without PRs?

@carlocab My bad, maybe it was homebrew-autoupdate people didn't use?

Yes, that one I recall no one raising their hands for.

Any idea why there's no results showing up in analytics here?

No 🥲

Would you be happy to take on any improvements to e.g. not committing data to repo without PRs?

Yep, happy to take on improvements, but I'd like some assistance with continuing maintenance (even if it's just poking me on Slack to fix things).

Any idea why there's no results showing up in analytics here?

No 🥲

@carlocab Which commands do you manually run and when, just so I understand?

Yep, happy to take on improvements, but I'd like some assistance with continuing maintenance (even if it's just poking me on Slack to fix things).

Yup, fair.
Will leave this open for now in case anyone else feels strongly it shouldn't be part of Homebrew.

Any idea why there's no results showing up in analytics here?

I think we only collect analytics for the new AbstractCommand subclasses?
https://github.com/Homebrew/brew/blob/92adf6a0b71e59d3348c982d11e2560dc9317687/Library/Homebrew/brew.rb#L88-L91
Homebrew/homebrew-command-not-found (and probably other official tap commands) have not been refactored.

Good catch, thanks @cho-m!

@carlocab Which commands do you manually run and when, just so I understand?

I rarely run it manually these days. When I do, it's usually to run brew which-formula because I want to know which formula provides a command.
However, I do have it set up as a command-not-found handler, via https://github.com/ohmyzsh/ohmyzsh/tree/5e59d21e5465ec0694441f41ff0acf1ddf4de187/plugins/command-not-found.

Would it be worth updating the command to the use the new DSL so we can collect stats before deprecating it?

Yes, I think this is the best thing for now.
Going to mark this as draft for now until we've done that.

I'm also using this. And of course, since most of the software I use is already installed by my dotfiles, command-not-found is very rarely actually called. command-not-found-init shouldn't do any network request given it's called when initializing a new terminal like shellenv.

Have opened PRs to gather analytics: https://github.com/Homebrew/homebrew-command-not-found/pull/156 (and others). Closing this until at least 30 days after merge so we can see what usage levels are like.


how to start the cmd file minimized?

hi, is there a way to start "Requestrr.WebApi.exe" minimized on windows startup? Since I use plex and radarr on my own computer I don't want to be bothered with the cmd file in my taskbar or when I alt tab between windows, thanks!

Hi @billfckme,
Currently in the program there isn't a way to start it minimized or in the background, however, you can use some Windows features to make this happen.
If your straight executing the EXE, you can create a shortcut and on the shortcut you can right click, go to "Properties" and set the shortcut to start minimized in the "Shortcut" tab.
If you want it to start in the background, you could setup a task to do this in "Task Scheduler", you should be able to make it run in the background there and not show on your desktop, however, I have not tried this...
On a side note, I would like to add the function to make Requestrr act as a Windows service in the future, so this might be the way to go when I have done so.
I hope that helps for the meantime.

Hi @billfckme,
Currently in the program there isn't a way to start it minimized or in the background, however, you can use some Windows features to make this happen. If your straight executing the EXE, you can create a shortcut and on the shortcut you can right click, go to "Properties" and set the shortcut to start minimized in the "Shortcut" tab. If you want it to start in the background, you could setup a task to do this in "Task Scheduler", you should be able to make it run in the background there and not show on your desktop, however, I have not tried this...
On a side note, I would like to add the function to make Requestrr act as a Windows service in the future, so this might be the way to go when I have done so.
I hope that helps for the meantime.

This app will turn any executable into a Windows service. You can even supply user credentials to the service so it will run without you needing to login https://nssm.cc/