The openapi.definition decorator struggles in a sync context

The openapi.definition decorator with a definitions.RequestBody instance or a non-empty dictionary as the body parameter transforms the target function to a coroutine.
For example, in my case, I'm using the decorator in a sync view...
@bp.get("/top/tracks")
@openapi.definition(
    body=oa.RequestBody(serializers.TopTracksRequest, required=True),
    summary="Fetch top tracks",
    response=[
        oa.Response(serializers.TopTracksResponse),
        oa.Response(
            Error, status=404, description="not found. note: sync is required."
        ),
        oa.Response(Error, status=400, description="bad request"),
    ],
)
@authenticate
def fetch_top_tracks(request):
    ...

...  that uses the Django ORM to retrieve data, and the only way I have to avoid the Django async safety check is to remove the definitions.RequestBody definition as a body (or perform an ugly workaround).

The code block that triggers the view actually detects the function as a coroutine and calls it as such
When I remove the body parameter, everything works as expected and the view is not triggered as a coroutine.
@bp.get("/top/tracks")
@openapi.definition(
    #body=oa.RequestBody(serializers.TopTracksRequest, required=True),
    summary="Fetch top tracks",
    response=[
        oa.Response(serializers.TopTracksResponse),
        oa.Response(
            Error, status=404, description="not found. note: sync is required."
        ),
        oa.Response(Error, status=400, description="bad request"),
    ],
)
@authenticate
def fetch_top_tracks(request):
    ...

I don't know if this behaviour is expected, my guess is that it has something to do with this check. Maybe it could be useful to add fully sync context support to this decorator, or add a flag to avoid explicitly this kind of behaviour.
Maybe it's totally expected, I'm missing the reasoning behind it and the body parameter is not expected to be used in a sync context (or needs some unknown -by me- configuration).
Anyway, if the behaviour is expected, I would love to know the reasoning behind it. Also, it could be useful to have it in the documentation since I couldn't understand what was happening easily. If it's already in the documentation and I couldn't find it; Send me a link! 😉
Thanks a lot!
Loving Sanic ❤

Thanks, definitely not intended and needs investigation and a fix.

So, I am not sure that this actually a bug here. This seems to work on both def and async def.
from sanic import Request, Sanic, json
from sanic_ext import openapi
from sanic_ext.extensions.openapi import definitions as oa

app = Sanic(__name__)

class Error:
    ...

class TopTracksResponse:
    ...

@app.get("/")
@openapi.definition(
    summary="Fetch top tracks",
    response=[
        oa.Response(TopTracksResponse),
        oa.Response(
            Error, status=404, description="not found. note: sync is required."
        ),
        oa.Response(Error, status=400, description="bad request"),
    ],
)
def handler(request: Request):
    return json({"foo": "bar"})

Looking at your exception, the problem looks to be something instead that Django has introduced. Namely, if there is a running event loop, then bail out:

It looks like they provide you with an escape hatch though:
from os import environ

environ["DJANGO_ALLOW_ASYNC_UNSAFE"] = True

I'll be honest, this is the first time that I have heard of someone using Django ORM inside Sanic. I am curious to know more about how you wired that up and your experience of and motivation for doing so.

Yeah, totally, the error itself triggers when Django ORM is run, but the idea is that I was able to use the Django ORM inside my views without any problems because the view wasn't triggered as a coroutine until I add the documentation decorator. So it was fully compatible with the Django ORM in many cases, but the decorator converted the view as a coroutine, so I had to add some patch for Django ORM. Anyway, totally agree that this is not a bug itself, meaning that it's just incompatibility (without a workaround) with some non-async projects.

I'll be honest, this is the first time that I have heard of someone using Django ORM inside Sanic. I am curious to know more about how you wired that up and your experience of and motivation for doing so.

The motivation of why we choose Django resides in various ideas; The most important is that Django 4.1 releases in a couple of months with a new Asynchronous ORM interface so it will be fully compatible with Sanic in every case. So at my opninion we're at doors to have projects with Sanic as REST Framework and Django as ORM, since -as my point of view- Sanic is way better as a REST framework, in many cases. Ours included.
Until now, we were working with Peewee but for many issues with it, we decide it that we're going to play safe and use Django since we had more experience and it's widely recognized. We love Sanic for its Flask-type design, and obviously, the amazing performance it has, so we are excited about this new Django upgrade and loving the idea of working with Sanic and Django comfortably.

For now, until this update comes, we're basically mantaining the Django ORM inside Sanic encapsulating some views and services inside the asgiref.sync.sync_to_async method - as it's well explained in the Django documentation. And as support, we have a little async Django manager to work with QuerySets asynchronously, but we use it just for little things, and our motivation to have it is to have our views async as much as we can. It is not an ideal scenario, but works for us and as I said is just a migration step before upgrading to Django 4.1.

Thanks for the information. I did not know about the ORM coming with an async version soon. Progress!

this is highly unsafe and not recommended on production environments since it leads to data loss

How so? All they are doing is checking if there is a loop. I am not sure that I see how this would lead to data loss.

Closing this as it is a Django issue.


Python 2.7.10 Mac OS X 10.10 - binascii.Error TypeError: Incorrect padding

I have the following code:
 key = 'my_storage_account_key'
 blob_service = BlobService(account_name='my_storage_account', account_key='key')
 container = u'test'
 containers = blob_service.list_containers()

which results in the following error:
 ---------------------------------------------------------------------------
 TypeError                                 Traceback (most recent call last)
 <ipython-input-15-5ecff0813140> in <module>()
       3
       4 container = u'test'
 ----> 5 containers = blob_service.list_containers()
       6 if container not in containers:
       7     try:

 /Users/emh5y7u/Documents/MIDS/W205/W205env/lib/python2.7/site-    packages/azure/storage/blob/blobservice.pyc in list_containers(self, prefix, marker, maxresults, include)
     291             request, self.use_local_storage)
     292         request.headers = _update_storage_blob_header(
 --> 293             request, self.authentication)
     294         response = self._perform_request(request)
     295

 /Users/emh5y7u/Documents/MIDS/W205/W205env/lib/python2.7/site-packages/azure/storage/blob/_serialization.pyc in _update_storage_blob_header(request, authentication)
      41     request.headers.append(
      42         ('Content-Type', 'application/octet-stream Charset=UTF-8'))
 ---> 43     authentication.sign_request(request)
      44
      45     return request.headers

 /Users/emh5y7u/Documents/MIDS/W205/W205env/lib/python2.7/site-packages/azure/storage/auth.pyc in sign_request(self, request)
      68             self._get_canonicalized_resource_query(request)
      69
 ---> 70         self._add_authorization_header(request, string_to_sign)
      71
      72     def _get_canonicalized_resource_query(self, request):

 /Users/emh5y7u/Documents/MIDS/W205/W205env/lib/python2.7/site-packages/azure/storage/auth.pyc in _add_authorization_header(self, request, string_to_sign)
      47
      48     def _add_authorization_header(self, request, string_to_sign):
 ---> 49         signature = _sign_string(self.account_key, string_to_sign)
      50         auth_string = 'SharedKey ' + self.account_name + ':' + signature
      51         request.headers.append(('Authorization', auth_string))

 /Users/emh5y7u/Documents/MIDS/W205/W205env/lib/python2.7/site-packages/azure/storage/_common_conversion.pyc in _sign_string(key, string_to_sign, key_is_base64)
      81 def _sign_string(key, string_to_sign, key_is_base64=True):
      82     if key_is_base64:
 ---> 83         key = _decode_base64_to_bytes(key)
      84     else:
      85         if isinstance(key, _unicode_type):

 /Users/emh5y7u/Documents/MIDS/W205/W205env/lib/python2.7/site-packages/azure/storage/_common_conversion.pyc in _decode_base64_to_bytes(data)
      71     if isinstance(data, _unicode_type):
      72         data = data.encode('utf-8')
 ---> 73     return base64.b64decode(data)
      74
      75

  /usr/local/Cellar/python/2.7.10_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/base64.pyc in b64decode(s, altchars)
      74     except binascii.Error, msg:
      75         # Transform this exception for consistency
 ---> 76         raise TypeError(msg)
      77
      78

 TypeError: Incorrect padding

I've copied the key correctly, I've tried base64 encoding the key but I can't seem to establish a connection to the storage account. I'm not sure what else to try...

You don't need to base64 encode the key -- just pass in directly what you got from the portal or wherever else you got the key from. I know you said you've copied correctly, but two things to confirm right off the bat since they're common errors:

Does the key you're passing have whitespace around it? If so, make sure you remove that.
Many keys have = signs in them or end in that character. If your key ends in symbols, they are indeed part of the key and should be included.

Tell me either way and if these aren't the issues we'll keep debugging!

I created a completely clean Python 2.7 virtualenv and reloaded the Azure SDK and Azure storage modules. All works ok now.


Awareness of .yarnrc

Stumbled upon this possibly being needed by someone in the future while debugging: https://github.com/yarnpkg/yarn/issues/2935
registry can also be set in .yarnrc and if someone is using the workaround for the above and store it in .yarnrc instead of .npmrc then this package may not pick that up. I haven't tested it yet though, just thought it might be helpful to have this issue here.

Related: https://github.com/yarnpkg/berry/issues/264


react: Uncaught TypeError: e.git is undefined

From https://github.com/cucumber/cucumber-jvm/issues/2464 given this meta message:
{
  "meta": {
    "protocolVersion": "17.1.1",
    "implementation": {
      "name": "cucumber-jvm",
      "version": "7.2.2"
    },
    "runtime": {
      "name": "OpenJDK 64-Bit Server VM",
      "version": "11.0.2+9"
    },
    "os": {
      "name": "Linux"
    },
    "cpu": {
      "name": "amd64"
    },
    "ci": {
      "name": "Jenkins",
      "url": "https://example.com",
      "buildNumber": "6"
    }
  }
}

The html report will fail to render with:
Uncaught TypeError: e.git is undefined
    CICommitLink file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    ni file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    Uo file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    bl file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    zl file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    El file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    pl file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    ll file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    Ql file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    es file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    ml file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    es file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    render file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    9963 file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    n file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    <anonymous> file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34
    <anonymous> file:///home/mpkorstanje/Downloads/HTML_20Report_206/index.html:34

Because the CICommitLink assumes git is never undefined.
https://github.com/cucumber/common/blob/0fd7c42c53420566a6b5e3bfc42565cf3457443b/react/javascript/src/components/app/CICommitLink.tsx#L24
However the message specification allows it to be undefined:
https://github.com/cucumber/common/blob/0fd7c42c53420566a6b5e3bfc42565cf3457443b/messages/jsonschema/Meta.json#L6-L11

@davidjgoss we have to update the code to anticipate ci.git being undefined. I believe I fixed this in https://github.com/cucumber/cucumber-react/commit/0445c6ef0165a2877af6de07e1f0334a5de53fe6#diff-3cf19d71bb3e6f73142532ec673421a65dba96ef99e21e777315dc4a70ccb5c6 - thanks to the stricter null-checking that I enabled after we moved Cucumber-React to a new repo.
So the question is - do we release a patch release from this repo (the code is still there), or do we push through with https://github.com/cucumber/cucumber-react/pull/11 so we can release from the new repo? How far off are we from making the new Cucumber-React releaseable?

@aslakhellesoy still a couple of issues over on that PR:

3 failing tests around search functionality
CI step for test hanging

I think if it's not too painful might be best to get a fix out from the monorepo today.

Ok makes sense. I'll try to get that done tonight or tomorrow.

What is the status of this issue?
Is it fixed?
The ESM thing will not be ready soon so if nothing else is in progress, we may release it?

@aurelien-reeves agreed, 🚀

@aslakhellesoy Should we release a RC as this will be a new major version?
I assume the releasing guide suggest to release RC with final cucumbers, not internals!?

As described in RELEASING.md:

If you are releasing cucumber-{jvm,js,ruby} and bumping the MAJOR version, make a -rc.N release candidate. The release candidate should be available for at least a month to give users time to validate that there are no unexpected breaking changes.

I'll update the text to explicitly state that no RC is needed for libraries, to remove any doubt.

Updated: https://github.com/cucumber/.github/commit/e78ca1dd66d327aa1a7d3b010d8ddc2a9866ba2f

Thanks.
I actually did that update myself: https://github.com/cucumber/.github/commit/83d26377ae7ce561d9efee6d3b50a75ea38ceb69
As I knew how to process to release a new RC version, I did even not scroll down to that section 😓

It should be fixed as part of<EMAIL_ADDRESS>

It doesn't break any more but, some information is still missing:
With this meta message:
{
  "meta": {
    "protocolVersion": "17.1.1",
    "implementation": {
      "name": "cucumber-jvm",
      "version": "${parent.version}"
    },
    "runtime": {
      "name": "OpenJDK 64-Bit Server VM",
      "version": "15.0.2+7"
    },
    "os": {
      "name": "Linux"
    },
    "cpu": {
      "name": "amd64"
    },
    "ci": {
      "name": "GitLab",
      "url": "https://gitlab.com/gitlab-examples/ci-debug-trace/-/jobs/379424655",
      "buildNumber": "12345"
    }
  }
}

I would expect to see the url to the job.

I've reported the issue here: https://github.com/cucumber/cucumber-react/issues/50
@mpkorstanje agree to close this one?


fix(ios): bring back resetting to original position on menu dismissal

This PR implements the solution solution proposed by @alfrol in https://github.com/mpiannucci/react-native-context-menu-view/issues/109. Very important thing
Thanks for the great library!

Thank you for implementing!!


Update code to make DRY

Check this out.

This needs to be tested with the full system. Confident of my work but never leave it to chance.

Line 245/246:
GPS_CONNECTED = true;
if (GPS_CONNECTED) {
This if block will always execute. If it's always supposed to execute, remove line 245 and the if check. If it's only supposed to execute when the condition is true, remove line 245. Either way, that line needs to go.

There are 3 sensor setup functions: set_up_gps, set_up_bmp, and set_up_bno. Each of them does different things, which make sense if each sensor has a different setup procedure. However, there's also a function check_for_bmp_status, which does the same thing for the BMP as set_up_bno does for the BNO. These functions ought to be named consistently.
Corollary to that, since both the BMP and BNO have a connection check with a timeout (in check_for_bmp_status and set_up_bno, respectively), these could both be done in the same loop. Should the GPS sensor have a similar connection timeout check?

One of the principles of functional programming is that a function does the exact same thing given the same inputs every time it is called. However, sample_data_and_build_json_packet does not do this. Its behavior depends on global variables, which it changes the value of.
In effect, this function is being used inside the setup function to determine whether it should do its job. It's its own validator. This is not ideal.
Instead, this function ought to be broken up into at least 3 different ones:

create_json_packet
sample_sensors
set_sensor_status_flags
send_data

Then setup isn't calling sample_data_and_build_json_packet to determine whether sample_data_and_build_json_packet should do its job.

Line 44/45 and 49/50:
double* gps_alti_offset_address = &gps_base_altitude; // Offset for the GPS for launch reference frame```
```double base_altitude = 0;                     // Memory location for the offset for the BMP
double* alti_offset_address = &base_altitude; // Offset for the base altitude for the BMP```
The second variable in either of these is the memory location, and the first is the offset. Comments ought to be corrected. Also, not entirely sure why both are needed; I'd appreciate an explanation.

Line 60: // Function definitions
These are function declarations, the definitions are at the bottom.

Lines 56/57:
imu::Vector<3> vaccel;        // 3-dimensional Vector acceleration (non-conservative)
imu::Vector<3> laccel;        // 3-dimensional Vector acceleration (conservative)

These variable names ought to be more descriptive. For example, accel_nc and accel_c would work.

Out of curiosity, would it be possible to split this into multiple scripts? For example, this one could be called main.ino, and contain just includes, setup, and loop. Sensor-specific setups could be contained in their respective scripts. JSON-handling functions could also go in their own script.
This would make it much easier for the next programmer to change and add on to.

Unfortunately, it cannot, as the boot loader for the Teensy will only accept one file unless we wrote a library for each, which is overkill for what this actually does, in my opinion.

Unfortunately, it cannot, as the boot loader for the Teensy will only accept one file unless we wrote a library for each, which is overkill for what this actually does, in my opinion.

I mean, at the top, we have includes, so the main script could include the rest. Is that prohibited?

Shouldn’t be. I’d have to go look at how it includes the binaries but I think we could should just add them in as libraries through the Arduino IDE and call them just like the other ones. I’ll look at it and comment here.

So the real issue with multiple files is that the IDE compiler doesn’t work like a normal GCC compiler and actually compiles code from a different directory. A lot of people have been looking for solutions but the limitations of the IDE are what they are. You can create a library but, again, for what we have it may be overkill unless we move to not doing JSON and want to do custom structures.

Merging for the sake of getting all this into one place for future review/use.


Metrics information extract

Metrics service like kube-metrics will expose a service to let Prometheus to know how the node / cluster works.
Its information is useful.
Current Idea is using grok expression to extract useful information about the cluster.
What's grok expression?

Grok is a regular expression dialect that supports reusable aliased expressions. Grok works really well with syslog logs, Apache and other webserver logs, mysql logs, and generally any log format that is written for humans and not computer consumption.

https://www.elastic.co/guide/en/elasticsearch/reference/current/grok.html
it's working the branch feat/kube-metrics

more about available kube state metrics https://github.com/kubernetes/kube-state-metrics/blob/main/docs/README.md


All languages (COBOL first) - Contextual property-groups

Description of the enhancement requested
Z Open Editor allows property-groups to be declared in the settings.json file or in the zapp.yaml file.
These property-groups serve among other things to search for COPYBOOKs / INCLUDEs on the remote mvs system.
The names of the libraries, (syslib), are not necessarily fixed, but can depend on a context: the user code (mvs), the Git branch (part of branch name), and any other context information.
Is there a way to introduce variables into the information managed by property groups?
The values of these variables could be declared elsewhere, either in the user's settings.json file, or in the workspace's settings.json, or through VS Code's context variables, or by dynamic calculation when searching.
Use case: we want to reference user-level libraries in property-groups, (&SYSUID..xxxx.COPYLIB).
Note: the userid is an information available in the z/OSMF (or RSE) profile.

After research (consultation of Json validation schema) property-groups support variables expressed in the format ${variable}.
I wanted to test with a variable in mvs format:
  - name: cobol-mvs
    language: cobol
    type: mvs
    syslib:
      - "&USERID..PUBLIC.COBOL.COPYLIB"
      - "DSGETU.PERMIBL.SIRIS.COPY"
      - "RSGREF.PERMIBL.SIRIS.COPY"

2022-05-16T12:55:47.477+0200 DEBUG: ZoweResourceResolver.checkMVSConnection: Running a Zowe profile connection test using the profile "GMVS" by reading data set "sys1.proclib".
2022-05-16T12:55:47.625+0200 DEBUG: ZoweResourceResolver.downloadMVSDataset(RequestId:klh10e): Looking for data set member "&USERID..PUBLIC.COBOL.COPYLIB(THECOPY)" on MVS using Zowe CLI Profile "GMVS".
2022-05-16T12:55:47.792+0200 DEBUG: ZoweResourceResolver.downloadMVSDataset(RequestId:klh10e): Data set member search request for "&USERID..PUBLIC.COBOL.COPYLIB(THECOPY)" returned:
>>> z/OSMF REST API Error:
Rest API failure with HTTP(S) status 500
category: 1
rc:       4
reason:   3
message:  Data set name is not valid
details:
  - IKJ56231I UTILITY DATA SET NOT ALLOCATED, SYSTEM OR INSTALLATION ERROR+
  - IKJ56231I TEXT UNIT X'0002' CONTAINS INVALID PARAMETER

[12:55:47.800144] ServletDispatcher[I]: Received request: GET /ds/&USERID..PUBLIC.COBOL.COPYLIB(THECOPY)
[12:55:47.800154] ServletDispatcher[I]: z/OSMF transaction: tx000000000000050B
[12:55:47.800178] ServletDispatcher[I]: Invoking Servlet: SELECT  PGM(IZUGGTDS) PARM(-LI,t -q 5505036 -r 0x29bf5690 )
[12:55:47.805809] GetDatasetServlet[E]: BPXWDYN allocation failed S99ERROR=0x035C S99INFO=0x0002
[12:55:47.805817] GetDatasetServlet[E]: BPXWDYN command was: "alloc da('&USERID..PUBLIC.COBOL.COPYLIB(THECOPY)')  SHR
gdgnt mount"
[12:55:47.805837] GetDatasetServlet[W]: IKJ56231I UTILITY DATA SET NOT ALLOCATED, SYSTEM OR INSTALLATION ERROR+
[12:55:47.805840] GetDatasetServlet[W]: IKJ56231I TEXT UNIT X'0002' CONTAINS INVALID PARAMETER

Property-groups support VS Code variable as ${varname}.
I tried using an mvs variable in a syslib name:
  - name: cobol-mvs
    language: cobol
    type: mvs
    syslib:
      - "&SYSUID..PUBLIC.COBOL.COPYLIB"
      - "DSGETU.PERMIBL.SIRIS.COPY"
      - "RSGREF.PERMIBL.SIRIS.COPY"

2022-05-16T13:14:58.808+0200 DEBUG: ZoweResourceResolver.checkMVSConnection: Running a Zowe profile connection test using the profile "GMVS" by reading data set "sys1.proclib".
2022-05-16T13:14:59.405+0200 DEBUG: ZoweResourceResolver.downloadMVSDataset(RequestId:c8802): Looking for data set member "&SYSUID..PUBLIC.COBOL.COPYLIB(THECOPY)" on MVS using Zowe CLI Profile "GMVS".
2022-05-16T13:14:59.610+0200 DEBUG: ZoweResourceResolver.downloadMVSDataset(RequestId:c8802): Data set member search request for "&SYSUID..PUBLIC.COBOL.COPYLIB(THECOPY)" returned:
>>> z/OSMF REST API Error:
Rest API failure with HTTP(S) status 500
category: 1
rc:       4
reason:   3
message:  Data set name is not valid
details:
  - IKJ56231I UTILITY DATA SET NOT ALLOCATED, SYSTEM OR INSTALLATION ERROR+
  - IKJ56231I TEXT UNIT X'0002' CONTAINS INVALID PARAMETER

[13:14:59.597630] ServletDispatcher[I]: Received request: GET /ds/&SYSUID..PUBLIC.COBOL.COPYLIB(THECOPY)
[13:14:59.597641] ServletDispatcher[I]: z/OSMF transaction: tx0000000000000513
[13:14:59.597664] ServletDispatcher[I]: Invoking Servlet: SELECT  PGM(IZUGGTDS) PARM(-LI,t -q 5570572 -r 0x29bf5690 )
[13:14:59.604326] GetDatasetServlet[E]: BPXWDYN allocation failed S99ERROR=0x035C S99INFO=0x0002
[13:14:59.604332] GetDatasetServlet[E]: BPXWDYN command was: "alloc da('&SYSUID..PUBLIC.COBOL.COPYLIB(THECOPY)')  SHR
gdgnt mount"
[13:14:59.604351] GetDatasetServlet[W]: IKJ56231I UTILITY DATA SET NOT ALLOCATED, SYSTEM OR INSTALLATION ERROR+
[13:14:59.604353] GetDatasetServlet[W]: IKJ56231I TEXT UNIT X'0002' CONTAINS INVALID PARAMETER
[13:14:59.604736] ServletDispatcher[I]: Servlet IZUGGTDS completed,  RC=0

TEXT UNIT X'0002' is dsname...
According to the documentation of the SVC 99, on the management of file names, it would be possible to use variables if the file name is specified between apostrophes:
https://www.ibm.com/docs/en/zos/2.2.0?topic=units-dsname-specification-key-0002
Try this:
  - name: cobol-mvs
    language: cobol
    type: mvs
    syslib:
      - "'&SYSUID..PUBLIC.COBOL.COPYLIB'"
      - "DSGETU.PERMIBL.SIRIS.COPY"
      - "RSGREF.PERMIBL.SIRIS.COPY"

Get this:
2022-05-16T13:25:59.667+0200 DEBUG: ZoweResourceResolver.downloadMVSDataset(RequestId:dldzi9): Looking for data set member "'&SYSUID..PUBLIC.COBOL.COPYLIB'(THECOPY)" on MVS using Zowe CLI Profile "GMVS".
2022-05-16T13:25:59.846+0200 DEBUG: ZoweResourceResolver.downloadMVSDataset(RequestId:dldzi9): Data set member search request for "'&SYSUID..PUBLIC.COBOL.COPYLIB'(THECOPY)" returned:
>>> z/OSMF REST API Error:
Rest API failure with HTTP(S) status 500
category: 1
rc:       4
reason:   3
message:  Data set name is not valid
details:
  - IKJ56231I UTILITY DATA SET NOT ALLOCATED, SYSTEM OR INSTALLATION ERROR+
  - IKJ56231I TEXT UNIT X'0002' CONTAINS INVALID PARAMETER

Try also:
  - name: cobol-mvs
    language: cobol
    type: mvs
    syslib:
      - "${env:USERNAME}.PUBLIC.COBOL.COPYLIB"
      - "DSGETU.PERMIBL.SIRIS.COPY"
      - "RSGREF.PERMIBL.SIRIS.COPY"

${env:USERNAME} not resolved...
2022-05-16T13:46:14.098+0200 DEBUG: ZoweResourceResolver.checkMVSConnection: Running a Zowe profile connection test using the profile "GMVS" by reading data set "sys1.proclib".
2022-05-16T13:46:14.286+0200 DEBUG: ZoweResourceResolver.downloadMVSDataset(RequestId:84zlm9): Looking for data set member "${env:USERNAME}.PUBLIC.COBOL.COPYLIB(THECOPY)" on MVS using Zowe CLI Profile "GMVS".
2022-05-16T13:46:14.318+0200 DEBUG: ZoweResourceResolver.downloadMVSDataset(RequestId:84zlm9): Data set member search request for "${env:USERNAME}.PUBLIC.COBOL.COPYLIB(THECOPY)" returned:
>>> EINVAL: invalid argument, mkdir 'C:/Trv/DepotsGIT/Temp/temp/_D_/GMVS/${env:USERNAME}.PUBLIC.COBOL.COPYLIB/'
<<<. If it is a Not Found error then no problem, will look in other property group locations.

Related to Use of symbols in dataset name or files for zowe/zowe-cli.
I think it will be difficult...


Der sarkissian2016

Pull Request
This PR is for a

[x] New Publication(s)
[ ] Correction

For the following list(s):

[ ] ancientmetagenome-anthropogenic (README)
[x] ancientmetagenome-environmental (README, ask @thedir-team-dirt for advice)
[ ] ancientmetagenome-hostassociated (README, ask @thedir-team-bugparty for advice)
[ ] ancientsinglegenome-hostassociated (README, ask @thedir-team-pathopeeps for advice)

New Publication
Publication Information
This pull request is to add samples from the following publication(s): DerSarkissian2016 (10.1111/1755-0998.12679)
This is to close #276
Checklist

[x] Publication is published (preprints currently not accepted)?
[x] Checked the publication is not already in the database?
[x] Checked samples in this publication are not previously published data (newly re-sequenced metagenomes are OK!)?
[x] Samples are shotgun metagenomes (hostassociated-singlegenome may also contain whole-genome enriched data)?
[x] Checked the list follows conventions as described in the corresponding sample type's README file (e.g. using ERS/SRS accession codes for ENA/SRA)?
[x] Changelog is updated to include the publication under 'Added'?
[x] Pull request has passed validation checks (see automated comment from github-bot)?
[x] Review requested from corresponding team (see list of lists above for each team)?
[x] (If applicable) Made a separate PR to add new categories to controlled lists (stored under /assets/enums, e.g. material or archive)? If so the corresponding PR is here: DerSarkissian2016 enum addition #319 2https://github.com/SPAAM-community/AncientMetagenomeDir/pull/319

If you do not know how to check errors in failed validation checks, expand here

Correction
This PR is for

[ ] ancientmetagenome-hostassociated
[ ] ancientsinglegenome-hostassociated
[ ] ancientmetagenome-environmental
[ ] ancientmetagenome-anthropogenic

Reference
This pull request is to correct samples from the following publication(s):
This is to close
Description
The issue is:

Checklist

[ ] Checked the corrected entries follow conventions as described in the corresponding sample type's README file (e.g. using ERS/SRS accession codes for ENA/SRA)
[ ] Changelog is updated to include the publication under 'Changed'?
[ ] Pull request has passed validation checks (see automated comment from github-bot)?
[ ] Review requested from corresponding team (see list of lists above for each team)?

I have a query about "feature" for this paper. It's an odd one and doesn't fit the environmental categories especially well, but I think it fits here better than our other streams.
There isn't much information about the samples, but they all come "from archaeological sites and collections". Many are modern, which I presume means modern zoological collections. For the few with very old ages, I've put "feature" as "archaeological site". I've provisionally put "Unknown" for the rest, but that's not an EnvO term and the value is mandatory, which is why it's failing checks.
Any suggestions?

Ooh, and I couldn't find the exact location of the site "Les Châtelliers du Vieil-Auzay", but a French speaker may be able to from the literature!

@Becky-Cribdon I think you can set feature to 'shell': https://www.ebi.ac.uk/ols/ontologies/envo/terms?iri=http%3A%2F%2Fpurl.obolibrary.org%2Fobo%2FUBERON_0006612
It's a valid ENVO term apparently: https://www.ebi.ac.uk/ols/search?q=shell&groupField=iri&start=0&ontology=envo

Thanks, James. "Shell" is a valid term that I'd used for material, but I guess it works for feature too. I've added it to the JSON (PR #322) and updated the table here.

Thanks, James. "Shell" is a valid term that I'd used for material, but I guess it works for feature too. I've added it to the JSON (PR #322) and updated the table here.

Yeah... but I tihnk it's the closest we can get. I just skim read the paper and it looks like there is a no one 'feature' so to say.

@Becky-Cribdon you're all set for merge! Thank you very much!

OK will do it to make sure it doesn't get left behind


Questions about feature

Hey,
I was wondering how the auto encode works, does it specifically try to hit a vmaf of 95 by default? Is there any option to hit the highest achievable, i.e. if a higher vmaf than 95 can he hit, then favour it?
Also, would it be possible to implement a batch/multi file processing on a folder basis, using walkdir with metadata recognition of video files? I'll look into the code to see if it doesn't add to much complexity.
I thought these two features would be a great addition, at least for my use cases.
What are your thoughts?

auto-encode/crf-search are indeed trying to find the highest crf value to satisfy the --min-vmaf and --max-encoded-percent (default 95 & 80 respectively).
You can configure these.
There was some discussion about handling multiple min-vmaf values in #49. The current solution is to try a high vmaf then if that fails try lower vmafs, caching makes this fairly efficient.
For recursive/multi-input there is no current support. But it's possible to do with a simple script, since (in general) ab-av1 encoding won't benefit from running concurrently.

Thanks for the reply!
Okay, so I just target with --min-vmaf.
I actually made a small rust app called ab-av1-walkdir that does that: https://github.com/BankaiNoJutsu/ab-av1-walkdir


It should be possible to build ordered-set without setuptools installed

python setuptools attempt to import ordered_set. Therefore it should be possible to build ordered_set with setuptools uninstalled. Perhaps try something like this:
try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup

Sorry, can you clarify? setuptools itself is depending on ordered-set?
Though it's hard to find clarity on exactly what a setup.py "should" contain, I believed I was following best practices here. And PEP 632 tells me that importing distutils is going to be deprecated in 8 months.

I took a look at setuptools to find that, in fact, the sdist command uses ordered-set. That's neat!
It also looks like setuptools has some machinery for including vendored dependencies that are copied into the setuptools directory, which seems like entirely the right way to do things so that not everything is a circular dependency.
Part of my motivation to keep ordered-set as a single Python module -- aside from just avoiding scope creep -- is that it can be very easily vendored into projects that can't use pip for whatever reason.
Is there some reason that this vendoring process isn't working in this case?

without setuptools:
❯ python3 -m pip install --upgrade ordered-set
Requirement already satisfied: ordered-set in /usr/local/lib/python3.9/site-packages (3.1)
Collecting ordered-set
  Using cached ordered-set-4.0.2.tar.gz (10 kB)
    ERROR: Command errored out with exit status 1:
     command<EMAIL_ADDRESS>-c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_0bec6915cb80478790fd0dcbf8a11a41/setup.py'"'"'; __file__='"'"'/private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_0bec6915cb80478790fd0dcbf8a11a41/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-pip-egg-info-tyobwwh0
         cwd: /private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_0bec6915cb80478790fd0dcbf8a11a41/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 20, in <module>
        from setuptools.dist import Distribution, Feature
      File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 34, in <module>
        from setuptools.depends import Require
      File "/usr/local/lib/python3.9/site-packages/setuptools/depends.py", line 7, in <module>
        from .py33compat import Bytecode
      File "/usr/local/lib/python3.9/site-packages/setuptools/py33compat.py", line 55, in <module>
        unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
    AttributeError: 'HTMLParser' object has no attribute 'unescape'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/f5/ab/8252360bfe965bba31ec05112b3067bd129ce4800d89e0b85613bc6044f6/ordered-set-4.0.2.tar.gz#sha256=ba93b2df055bca202116ec44b9bead3df33ea63a7d5827ff8e16738b97f33a95 (from https://pypi.org/simple/ordered-set/) (requires-python:>=3.5). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached ordered-set-4.0.1.tar.gz (10 kB)
    ERROR: Command errored out with exit status 1:
     command<EMAIL_ADDRESS>-c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_d42f18c5cac948d9b4d217345499d0df/setup.py'"'"'; __file__='"'"'/private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_d42f18c5cac948d9b4d217345499d0df/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-pip-egg-info-z80mqe5s
         cwd: /private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_d42f18c5cac948d9b4d217345499d0df/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 20, in <module>
        from setuptools.dist import Distribution, Feature
      File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 34, in <module>
        from setuptools.depends import Require
      File "/usr/local/lib/python3.9/site-packages/setuptools/depends.py", line 7, in <module>
        from .py33compat import Bytecode
      File "/usr/local/lib/python3.9/site-packages/setuptools/py33compat.py", line 55, in <module>
        unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
    AttributeError: 'HTMLParser' object has no attribute 'unescape'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/89/5d/60db38c980730fc91a0a9cf4c2b932037eecfbd6926aa2ebf450247c0096/ordered-set-4.0.1.tar.gz#sha256=a31008c57f9c9776b12eb8841b1f61d1e4d70dfbbe8875ccfa2403c54af3d51b (from https://pypi.org/simple/ordered-set/) (requires-python:>=3.5). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
  Using cached ordered-set-3.1.1.tar.gz (10 kB)
    ERROR: Command errored out with exit status 1:
     command<EMAIL_ADDRESS>-c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_0fdbfb032fb941698e9861274b68c590/setup.py'"'"'; __file__='"'"'/private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_0fdbfb032fb941698e9861274b68c590/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-pip-egg-info-1m4boz71
         cwd: /private/var/folders/jz/gh2db8td29q2prqyhbzsbf3h7xy09n/T/pip-install-783vx4j3/ordered-set_0fdbfb032fb941698e9861274b68c590/
    Complete output (11 lines):
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/usr/local/lib/python3.9/site-packages/setuptools/__init__.py", line 20, in <module>
        from setuptools.dist import Distribution, Feature
      File "/usr/local/lib/python3.9/site-packages/setuptools/dist.py", line 34, in <module>
        from setuptools.depends import Require
      File "/usr/local/lib/python3.9/site-packages/setuptools/depends.py", line 7, in <module>
        from .py33compat import Bytecode
      File "/usr/local/lib/python3.9/site-packages/setuptools/py33compat.py", line 55, in <module>
        unescape = getattr(html, 'unescape', html_parser.HTMLParser().unescape)
    AttributeError: 'HTMLParser' object has no attribute 'unescape'
    ----------------------------------------
WARNING: Discarding https://files.pythonhosted.org/packages/a3/b7/d4d69641cbe707a45c23b190f2d717466ba5accc4c70b5f7a8a450387895/ordered-set-3.1.1.tar.gz#sha256=a7bfa858748c73b096e43db14eb23e2bc714a503f990c89fac8fab9b0ee79724 (from https://pypi.org/simple/ordered-set/) (requires-python:>=2.7). Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

after:
❯ python3 -m pip install --upgrade setuptools
Requirement already satisfied: setuptools in /usr/local/lib/python3.9/site-packages (41.0.1)
Collecting setuptools
  Downloading setuptools-57.0.0-py3-none-any.whl (821 kB)
     |████████████████████████████████| 821 kB 921 kB/s
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 41.0.1
    Uninstalling setuptools-41.0.1:
      Successfully uninstalled setuptools-41.0.1
Successfully installed setuptools-57.0.0

❯ python3 -m pip install --upgrade ordered-set
Requirement already satisfied: ordered-set in /usr/local/lib/python3.9/site-packages (3.1)
Collecting ordered-set
  Using cached ordered-set-4.0.2.tar.gz (10 kB)
Building wheels for collected packages: ordered-set
  Building wheel for ordered-set (setup.py) ... done
  Created wheel for ordered-set: filename=ordered_set-4.0.2-py2.py3-none-any.whl size=8218 sha256=a1160e44068792d94ef529858c4dfefa582b3f86b7b05352b5d2d7f8432f753c
  Stored in directory: /Users/amirtal/Library/Caches/pip/wheels/c0/a1/09/0c736ab4beb6a48ae2f5f2529b2ad3770c8e6cc32d01d75a2c
Successfully built ordered-set
Installing collected packages: ordered-set
  Attempting uninstall: ordered-set
    Found existing installation: ordered-set 3.1
    Uninstalling ordered-set-3.1:
      Successfully uninstalled ordered-set-3.1
Successfully installed ordered-set-4.0.2

Taking a look at this again.
I don't understand why the "without setuptools" version has files such as /usr/local/lib/python3.9/site-packages/setuptools/py33compat.py, indicating that setuptools is installed. Could this be a conflict involving multiple Python installations? It doesn't look to me like it's connected to anything in ordered-set.

cannot re-produce when using stock python image (python:3.9.7).
so it might be related to conflicting python versions.
❯ docker run -it python:3.9.7 /bin/bash
Unable to find image 'python:3.9.7' locally
3.9.7: Pulling from library/python
955615a668ce: Pull complete
2756ef5f69a5: Pull complete
911ea9f2bd51: Pull complete
27b0a22ee906: Pull complete
8584d51a9262: Pull complete
524774b7d363: Pull complete
af193b9b3d11: Pull complete
aacb0e56e8f3: Pull complete
46cd7abc9e93: Pull complete
Digest: sha256:e6654afa815122b13242fc9ff513e2d14b00548ba6eaf4d3b03f2f261d85272d
Status: Downloaded newer image for python:3.9.7
root@f801b7daa5dc:/#
root@f801b7daa5dc:/#
root@f801b7daa5dc:/#
root@f801b7daa5dc:/# python3 -m pip install --upgrade ordered-set
Collecting ordered-set
  Downloading ordered-set-4.0.2.tar.gz (10 kB)
Building wheels for collected packages: ordered-set
  Building wheel for ordered-set (setup.py) ... done
  Created wheel for ordered-set: filename=ordered_set-4.0.2-py2.py3-none-any.whl size=8219 sha256=70658ce1746193908d1daf80404078c92eedd04ad413e76bd943463eec4b30ad
  Stored in directory: /root/.cache/pip/wheels/c0/a1/09/0c736ab4beb6a48ae2f5f2529b2ad3770c8e6cc32d01d75a2c
Successfully built ordered-set
Installing collected packages: ordered-set
Successfully installed ordered-set-4.0.2
root@f801b7daa5dc:/#

I'm looking into using Poetry as the primary way of building ordered-set, which would solve this problem.


Allow configuring the min gas price for vald from config or env var

Description/Reasoning
vald sets a default value for the gas price to use, and only allows overriding this via a flag. This is cumbersome to change. So, we should allow it to be configurable via the config file or an env var.
Current Behaviour
Expected Behaviour
Steps to reproduce (for bugs)
Relevant Logs or Files

@milapsheth this is done right?

#1322


Add documentation

In this PR

Sphinx documentation files are added for automatic generation using github actions, @judithabk6 we will have to configure the gh page hosting together as you are the owner of the repo
A template for showing examples has to be developed
Other modifications regarding documentation might be done here

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 86.53%. Comparing base (43c118b) to head (e748ef6).

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@           Coverage Diff            @@
##           develop      #84   +/-   ##
========================================
  Coverage    86.53%   86.53%
========================================
  Files            6        6
  Lines          750      750
========================================
  Hits           649      649
  Misses         101      101

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

LGTM so far, but the doc does not build yet on my box.

I didn't pushed the generated doc files as it will be done by the github action on the gh-pages branch (to host the doc online). This branch will be created when we'll configure it with @judithabk6 (the only one who can do it as the owner of the repo)

OK thx.
Still, we should be able to build the doc locally ?

Yes by running the following command in the docs folder : make html

Works on my box.


Change feature state for kms provider to beta.

KMS Provider will be graduating to beta in v1.12, reflecting this change on the website.

/lgtm

@immutableT @tengqm Now that this feature is going beta, does it need enhanced documentation?

@zparnold I think the current doc looks good.


Hide gallery section if no exhibits are configured

Reference Issues or PRs
Accompanies https://github.com/nebari-dev/jupyterlab-gallery/pull/20
What does this implement/fix?
Put a x in the boxes that apply

[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds a feature)
[ ] Breaking change (fix or feature that would cause existing features not to work as expected)
[ ] Documentation Update
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no API changes)
[ ] Build related changes
[ ] Other (please describe):

Testing

[ ] Did you test the pull request locally?
[ ] Did you add new tests?

Any other comments?

I'm not sure why CI failed here, I re-triggered it.


Add 🇷🇺 translations

Fully translated to Russian
So that people from all around world can use this beautiful plugin

Thank you


GitHub Actions: Reinstate fossa.com integration

Description of issue or feature request:
A recent switch from Travis (and Appveyor) to GitHub Actions removed fossa.com integration from our CI setup. (see #1242)
Current behavior:
No publication of license data on fossa.com
No fossa.com badge
Expected behavior:
Re-instate publication of license data on fossa.com
Re-add fossa.com badge (revert 0ab9ee7)

Copying discussion from duplicate:
me:

There does not seem to be anything preventing enabling FOSSA now: GH actions is directly supported via an OAuth GitHub App.
Maybe it makes sense to evaluate the need for FOSSA before we enable it though:

dependency updates are handled by dependabot
dependabot also does vulnerability alerts (based on CVEs and advisories) for the pip ecosystem

That leaves ... license compliance checks? is there something else that fossa provides?

lukas

IIRC fossa is or at least was a requirement for the https://bestpractices.coreinfrastructure.org gold badge.

I had a look and I think the CII requirements do not ask for specific tools (and fossa is not mentioned in the TUF entry). The analysis section should be relevant: https://bestpractices.coreinfrastructure.org/en/projects/1351#analysis
I want to mention that I was honestly asking about the FOSSA features -- I'm not familiar with it and their documentation is a bit vague so I'm not sure if it does something we'd be interested in.

"The project MUST have FLOSS automated test suite(s) that provide at least 90% statement coverage if there is at least one FLOSS tool that can measure this criterion in the selected language"

unittest + coverage fills this requirement, no?

🤦  I completely misread that sentence. For some reason I thought that "FLOSS automated test suite" was referring to license scanning.

Maybe @JustinCappos remembers why this was enabled in late 2017? (See 600e25f5f9e3e28bbd94f39bd9fd27d83d869563).


Patch Curl: CVE-2021-22898 and CVE-2021-22901

Merge Checklist

All boxes should be checked before merging the PR (just tick any boxes which don't apply to this PR)

[ ] The toolchain has been rebuilt successfully (or no changes were made to it)
[x] The toolchain/worker package manifests are up-to-date
[x] Any updated packages successfully build (or no packages were changed)
[x] All package sources are available
[x] cgmanifest files are up-to-date and sorted (./cgmanifest.json, ./toolkit/tools/cgmanifest.json, ./toolkit/scripts/toolchain/cgmanifest.json, .github/workflows/cgmanifest.json)
[x] LICENSE-MAP files are up-to-date (./SPECS/LICENSES-AND-NOTICES/data/licenses.json, ./SPECS/LICENSES-AND-NOTICES/LICENSES-MAP.md, ./SPECS/LICENSES-AND-NOTICES/LICENSE-EXCEPTIONS.PHOTON)
[x] All source files have up-to-date hashes in the *.signatures.json files
[ ] sudo make go-tidy-all and sudo make go-test-coverage pass
[x] Documentation has been updated to match any changes to the build system
[x] Ready to merge

Summary

What does the PR accomplish, why was it needed?
Change Log

Change
Change
Change

Does this affect the toolchain?

YES
NO
Associated issues

#xxxx

Links to CVEs

https://nvd.nist.gov/vuln/detail/CVE-YYYY-XXXX

Test Methodology

Pipeline build id: xxxx

Is this based on a correct branch?

Asking since there are 250+ modified files... :)


[feature request] use ngx-translate in nodejs

we would like to use ngx-translate on nodejs server to generate internationalized pdf on server, is it possible to use ngx-translate on server side with nodejs?

Why aren't you looking for a node specific solution? Like i18n-node for instance? ngx-translate is, like the name suggests, an Angular library. So as long as you aren't rendering your pdf content with Angular, I don't think this library can be of much help for you.

If you're looking to use it with angular universal (server side) then yes it works

just would like to share the same translations file beetween angular client side and server side

You'll have to provide a loader for the server side that works with nodejs, like this one: https://gist.github.com/ocombe/593d21598d988bf6a8609ba5fc00b67e
And use your usual loader client-side

yes but how to instanciate ngx-translate out of angular in node?

ah that's not possible, it only works if you use Angular Universal on the server side

is there a node specific module fully compatible with ngx-translate format ?

Not that we're aware of. The most used/famous one in node-land is probably i18n. You might be able to configure it to behave the same way but I'm not sure about it.

is there a node specific module fully compatible with ngx-translate format ?
@atiertant
Hello, am looking for the same thing, did you find something.
thnx


Added DeezNuts Token

@DeezNutsToken can you lowercase the name?

Also, you can use these graphics to say that your ERC20 token fully compatible with Trust Wallet (https://trustwalletapp.com/logo-assets). Tokens will show up automatically once the user receives them.
Feel free to share this on social media and tag Trust Wallet:
Twitter: @TrustWalletApp
Facebook: @TrustWalletApp


Include CSS variables for the different interaction states for color

From a suggestion from @seejamescode, it'd be great if we could include the interaction states for color as part of our output in SCSS, CSS, and JS variables for folks to use 👍

Perhaps and opt-in but not needed for Carbon X

Weighing in - Opt-in sounds nice. IIRC we have seen a performance issue by excessive usage of CSS variables a half year ago.


Problems with chrome

Using em Firefox

Now in Chrome

In chrome/chromium browser it's not working properly. All the shortcuts work, but I can't see the profile menu. It's just happen in chrome/chromium. Using firefox it's works fine.
Do you want more information, logs? let'me know.
thanks

Interesting... There was another user with the same problem but I was not aware that it occurs only in Chrome (I had tested in Safari and everything was working).
The error resides in the Firebase JS client library. Probably Firebase has changed something recently. I should update it. But maybe other problems will arise if I do so because currently I am using a two-year-old version! :smile:


Dynamic route table in HTTP connection manager

I would like to apply custom routing mechanism, where unknown routes are inspected mid-flow and then by some custom logic, there would be a routing rule to decide to which upstream cluster to send the request to.
For example, say that there's a pseudo-function that has the following logic:
string chooseCluster(string path) {
  if (MeetsInternalBusinessLogic(path))
    return "clusterA";
  else
    return "clusterB";
}

The implementation of "MeetsInternalBusinessLogic" is not something that can be managed statically, otherwise I would have manually configured the .yaml by known routes.
I read the documentation and wanted to make sure I follow the correct path here. Can the Virtual host's "matcher" config help here? Would it be possible to write a custom "matcher" that would take the HTTP ":path" as input, and return the required upstream cluster for that HTTP request? Is there any other way to implement this functionality?
Thanks for helping

May be a custom inline_cluster_specifier_plugin implementation could be helpful? This extension could be used to get upstream cluster name according to the HTTP request headers.

Thanks, is there an example or documentation that you can share regarding this?


[Feature request] Wayland backend

Please make the app use a wayland compatible (No XWayland)
(idk if dotnet has a wayland backend. This app https://flathub.org/apps/io.github.onionware_github.onionmedia also seems to not have a wayland backend)

It wouldn't be a dotnet limitation but a limitation with Avalonia (the UI framework SFP uses). It's possible that this has changed in upstream Avalonia though, it's been a while since I've looked at all of the changes they've made.


Moon Activity: fix css in chrome mobile version

The user interface (UI) of MoonActivity on the Chrome mobile version needs improvement.
Image:

No, it's not a real use case. See https://github.com/llaske/sugarizer/issues/422
Avoid sending a PR without having validation on the issue.

Thank you for the clarification.


Added Carthage support and possibility to initialize the tab bar in code.

If the code approach for initializing the tab bar is not sufficient please let me know.

thanks


[Add] dotnet resources

Description
Hi,
There are nuggets packages that I use on daily basis.
Like

MvvM Toolkit community
Entity Framework Core
Dependency Injection for both previous
Twilio

Regards,
Mafyou.
Screenshots
No response
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I am willing to work on this issue (optional)
[ ] I am a GSSoC'23 contributor

Hi @Mafyou! How can LinksHub benefit from these packages?

They are open sources and help developers. Don't Links Hub made this?
Sorry if I misunderstood

They are open sources and help developers. Don't Links Hub made this?
Sorry if I misunderstood

No you’re good @Mafyou! :) I just wanted more inquiry.

@Mafyou are you still working on the issue?

I'm currently in holiday. Next week update!

I'm currently in holiday. Next week update!

ok, thanks

Here are some update.
Do you want more ?

where?
please add a pr with updates if any

Here are some update. Do you want more ?

Where 😂😂

@Mafyou any updates please?

Why updates ? You are referencing librairies so do I.

@Mafyou are you done with the issue so plz add a pr or else we will close this.

👌

okay then closing it


Remove references to src-tauri

There's also a section in the readme, but I'm not sure what to update it to.

Looks fine to me


style(Table): remove hover on zebra colored row

table with zebra enabled made zebra colored rows white on hover. Removed this.
This does not need a changeset, since the changes to table has not been pushed to next yet

table with zebra enabled made zebra colored rows white on hover. Removed this. This does not need a changeset, since the changes to table has not been pushed to next yet

Its removed in comparison to main so shouldn't it be a changeset then?

table with zebra enabled made zebra colored rows white on hover. Removed this. This does not need a changeset, since the changes to table has not been pushed to next yet

Its removed in comparison to main so shouldn't it be a changeset then?

no, this was an expected change in a past PR about removing hover on table. This had a changeset in that PR

table with zebra enabled made zebra colored rows white on hover. Removed this. This does not need a changeset, since the changes to table has not been pushed to next yet

Its removed in comparison to main so shouldn't it be a changeset then?

no, this was an expected change in a past PR (#2285) about removing hover on table. This had a changeset in that PR

ah, this was missed when we removed hover?

table with zebra enabled made zebra colored rows white on hover. Removed this. This does not need a changeset, since the changes to table has not been pushed to next yet

Its removed in comparison to main so shouldn't it be a changeset then?

no, this was an expected change in a past PR (#2285) about removing hover on table. This had a changeset in that PR

ah, this was missed when we removed hover?

yes, this is a fix for this, Would have made a changeset if we had released a next version in between this and that PR :)


Can't access uploaded image

Recently day, I can't access to all of my uploaded file and they are no signal what's going on although I don't know how to inform an email directly.

The file table links is now fixed. I'm guessing this was the reason for you getting a url to our test bucket. If not, reopen

This is happening to me too


Password?????

what does it mean by password? what is the password? please help

wait nvm i can just go to protected.html and it works

killsecurlyfoundation is the password btw

wait nvm i can just go to protected.html and it works

Yeah. It's just a way to deter off people that won't actually try to use ONC properly.


Database query limitations

As of now, it is very possible to query a complete list of all videos from the database. In fact, the front page does so, but limits the results to a maximum of 60. That makes it trivial to find all videos at once, so it would be relatively easy for an attacker to scape our video database.
I don't know much about SQL, so there isn't a lot I can say about implementing this, but there must be limits to one's ability to query large amounts of information. The application servers should not have the ability to request mass tables of information on videos or users, like it could on the homepage.

This isn't relevant to the web app code, and has been moved to a table containing our tasks to harden the network.


cmd + z yield weird error

It says something like 2020-10-28T09:33:02.116Z undefined

I get those on a regular basis without any clear path for reproduction. It once even resulted in n endless loop of these popups being generated.
https://user-images.githubusercontent.com/5774545/103438424-4d5e4980-4c33-11eb-995d-2774086db3f1.mov

I get those on a regular basis without any clear path for reproduction. It once even resulted in n endless loop of these popups being generated.
https://user-images.githubusercontent.com/5774545/103438424-4d5e4980-4c33-11eb-995d-2774086db3f1.mov

Please enable debug logging and reproduce with it

Please enable debug logging and reproduce with it

The endless loop of toasts happened again. I had debug logging enabled to catch some messages but no output was written. Are the debug logs somewhere else?


clarify fields and attributes

this clearly denotes what constitutes a field/attribute and then
re-uses that reference when describing how sparse fieldsets
should operate.

This also provides an example of how the document could be inter-linked. Assuming ya'll like this, I'm going to put a manifest of terms and links at the bottom of the doc and link the hell out of everything.

I like the new subsections for Attributes etc., but I think the special mention about id, type, and self in Sparse Fieldsets would have been useful. It was a deliberate addition after all. A non-normative note would be enough, though.

Oh yeah and :+1: for the reference-style Markdown links. I was going to do the same thing. The spec could use more intra-doc linking, particularly where a term is used before it has been properly introduced.

@tkellen Sorry to bug you about this, but may I suggest simply:

Note: Only fields are affected. type, id, and self are included as normal.

No bugging, that's much better.

Updated!

Yay!
One more thing: in Complex Attributes, the definition includes "[complex attribute]", but the actual link is [complex attributes] in plural, so the link doesn't format.
Other than that, this looks good to me.

Great job at consolidation and clarification here, @tkellen!

Hmm... I gotta say, I kinda liked the original definition of a complex attribute that specifically said it can be anything. This has not been entirely clear to users.
In case the change has to do with accommodating the Markdown link, it's possible to use reference-style links even if the text doesn't match the link title:
If the value of an attribute is a JSON object or array, the member is called a [complex attribute][complex attributes]. The value is allowed to be any valid JSON structure.
As for the note, I figured "as normal" gives it away, but might as well make it foolproof:

type, id, and (optionally) self are included as normal

type, id, and (optionally) self are included as normal

Yes, that works for me. It is an important note.

I've updated that note.  @bintoro can you suggest a rewriting of that entire section if you feel it is unclear? I think it is fine, and I'm not sure how to incorporate your suggestion in a sensical way.

@tkellen I thought it was just fine earlier when I noticed the link wasn't being formatted :smile:
If the value of an attribute is a JSON object or array, the member is called a
["complex attribute"][complex attributes]. The value is allowed to be any valid
JSON structure. However, an object that constitutes or is contained in a complex
attribute must reserve the `id`, `type`, `links`, and `meta` members for future
use.

The statement "is allowed to be any valid JSON structure" was a deliberate addition. The word "structure" was meant to imply that you can have nested objects too. If it doesn't accomplish this, I can reword.
But, if you think it's unnecessary to mention that at all, then the current revision is good to go as is!

Updated!  Can you take another look?

Thanks @tkellen! That should make it clear.

:+1:


doc: website jobset url

What type of PR is this?
/kind documentation

What this PR does / why we need it:

fix website url
%w can be used only once in a format string

Which issue(s) this PR fixes:

Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?

None

Please split this in two PRs, as the changes are targeting completely unrelated things.
Also, we would like to cherry-pick the changes to the website without cherry-picking the other.

@alculquicondor Thanks for the suggestion. done!

@alculquicondor Thanks for the suggestion. done!

@googs1025 Aldo meant that we should split this PR into 2 separate PRs (code changes and document changes)

/cherry-pick website


Azure ML : Unable to write data in a job

Azure documentation issue guidance
Thanks for opening an issue in the Azure technical documentation repository.
We use GitHub issues as the primary channel for customer and community feedback about the Azure documentation.
Creating an issue
We prefer that you create documentation feedback issues using the Feedback link on the published article - the feedback control on the doc page creates an issue that contains all the article details so you can focus on the feedback part.
You can also create a feedback issue here in the repo. If you do this, please make sure your issue lists:

[ ] The relevant Azure service or technology.
[ ] A link to the published documentation article that you have feedback about.
[ ] Clear, specific feedback that the author can act on.

Pull requests and article contributions
If you know the change that is needed in an article, we encourage you to submit the changes directly using a pull request. If the change is large, or if you want to contribute an entire article, follow these guidelines:

[ ] Don't surprise us with a big pull request or a pull request with a new article! Submit an issue that describes the details of the proposed large change or new article.
[ ] Include the service or technology area.

We'll route the issue to the appropriate content team for review and discussion.
Tech support and product feedback
If you would like to contact Microsoft about other things, such as product feedback or tech support, please review these guidelines:

If you need technical support using Azure, the paid and free support options are described here: https://azure.microsoft.com/support/options/.

Each article in the Azure technical documentation contains a product feedback button - it's best to submit product feedback directly from a relevant article. Otherwise, you can submit product feedback for most Azure products in the following product feedback forum: https://feedback.azure.com/d365community/forum/79b1327d-d925-ec11-b6e6-000d3a4f06a4.

@ankitgupta1729
It would be great if you could add a link to the documentation you are following for these steps? This would help us redirect the issue to the appropriate team. Thanks!!

Hi @YashikaTyagi-MSFT, Thanks for asking. I was following this documentation and was unable to write data into a file for my working directory. Here, I have opened the issue because I have not found the solution in the documentation which I followed and also on the web.
But later I though that it is a technical issue, not a feedback issue and should not be asked here, so I asked it here
Sorry for the inconvenience. I am closing it.


Add doc comments for initializers for literals [NFC]

Types that conform to protocols for color, image, and file literals expose a public initializer that should not be called by users. However, the initializer lacks documentation to that effect. This PR adds that documentation.

@dabrahams As the original author of other doc comments in this file, does this look OK to you?

@xwu it's OK with me. @natecook1000 you?

Changes look good! Would you mind updating the protocol descriptions to match the rest of the ExpressibleBy___ gang as well? We're using "A type that can be..."

@swift-ci please smoke test

Nope, doesn't look like I have permissions :P

@swift-ci Please smoke test and merge

@swift-ci Please smoke test and merge


fix(pipeline): namespace not being detected

namespace wasn't being detected during CodeArtifact npm publish

funny pr


Add reporting functionality to dashboard

In order to enrich the data in the dashboard, we need to send in some data that is only available on the device(s). We do however need to do this in a secure fashion. Here's how I am thinking this would work as of right now:

We create a new API end-point in the dashboard using DRF
We then do a simple POST to this end-point periodically from the device (TBD how we best run this)

While this all sounds simple, the devil is in the details. Credentials will be the trickiest part to solve, but here's how I'm currently envisioning it working:

We create a new authentication method for DRF that can take the RPI serial as the username and a token as the password

This set of credentials will only be able to update the status for itself (i.e. match on serial number)

We then create a Celery task that generates these sets for all devices
As part of the above Celery task, we also create a new environment variable for the particular device with the token (e.g. DASHBOARD_TOKEN).

If we ever wanted to rotate the credential for a given device, we simply remove the environment variable and the device Celery task will then auto-populate it again.

On the device side, we simply use the RPI serial and the new environment variable as the credentials when submitting the POST

@KevinWassermann94 guess we had solved this in other ways so probably no longer relevant?


feature(metrics): metrics for aggregation

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.
What's changed and what's your intention?
Add similar metrics in Join to Agg:

lookup miss
total lookup
total number of keys being cached.

Checklist

[ ] I have written necessary rustdoc comments
[ ] All checks passed in ./risedev check (or alias, ./risedev c)

Documentation
Refer to a related PR or issue link (optional)

let me also modify the dashboard...


Include function URL in logs

Love this extension! Thank you for creating it.
Right now when publishing a new function to Azure the logs will give you the link to the function app but not for the individual functions. So this means that in order to test that my function has been deployed successfully I still need to go open the portal and copy manually the URL. Would be great to have the function url included in the logs.

@simonaco I assume you're only interested in the url for Http triggers? Or do you find the other urls useful?

@EricJizbaMSFT yes, I can only see use cases for the HttpTriggers urls.


2b. Prevent watch channel deadlocks in the state

Motivation
This PR makes watch channel deadlocks in the state impossible, which avoids time-consuming investigations into deadlock causes.
Specifications
If the watch receiver holds a read lock, then the sender tries to get the write lock, then another receiver tries to get another read lock, the watch channel can deadlock.
See the "potential deadlock example" in:
https://docs.rs/tokio/latest/tokio/sync/watch/struct.Receiver.html#method.borrow
Designs
The WatchReceiver wrapper clones the watched data before allowing access to it. This prevents read-write-read deadlocks, because the read lock is dropped immediately after the clone.
Solution
Deadlock prevention:

Add a WatchReceiver wrapper that always clones the borrowed watch data

Memory optimisations:

Drop the shared Arcs as quickly as possible

Ergonomics:

Make read::block more flexible, by accepting any AsRef
Make the block method docs consistent

Review
This is part of an urgent series of state refactor PRs, which will cause merge conflicts with other PRs.
This PR depends on PR #3847.
Reviewer Checklist

[ ] Code implements Specs and Designs
[ ] Tests for Expected Behaviour
[ ] Tests for Errors

Follow Up Work
The rest of the PR series.

@mergifyio refresh

Looks ok to me but I kinda would like @jvff to look at it.


[casr-cluster] Support cluster merging

Add ---cluster-merge option or update --merge one for casr-cluster to support cluster merging

An idea is to combine/merge sets of clusters or their demdrograms from two different casr-libfuzzer/casr-alf runs with overlapping to get single set of clusters with saving old casreps and containing new unique ones.


Present new binary for checking hashes

Closes #2348

Should be working, but maybe I missed some use-case or something else.
For external buckets use: something like this

Should be ready to merge 🎉


Updates run_lora_clm.py with enhanced dataset support

This is a PR on my fork for peer review before submitted to the upstream repo. This script previously supported two datasets from Hugging Face hub for non-SQL prompt situations: tatsu-lab/alpacaand timdettmers/openassistant-guanaco. The timdettmers/openassistant-guanaco dataset only has one column named text, whereas the tatsu-lab/alpaca has the standard instruction, input, and output columns.
To support more datasets, I added arguments to allow the user to specify column names for "instruction", "input", and "output" (or in the case of using SQL prompts it's "question", "context", and "answer"). If the user provides custom column names, I am renaming those columns to use the standard names (instruction/input/output or question/context/answer) so that the rest of the code in the script will work as expected. Also, for non-SQL prompt datasets, the "input" is optional (which is why there's a "prompt_with_input" and "prompt_without_input" template), but the existing code was handling the situation where the "input" column was blank. I updated the code to also handle the "input" column not existing at all.
I left the code specific to timdettmers/openassistant-guanaco alone, since that was a more special use case where there is only one column with already formed prompts. If there was interest in generically handling more datasets like this one, I could look into that in a follow up PR.
I tested this update with several different datasets: databricks/databricks-dolly-15k, ruggsea/stanford-encyclopedia-of-philosophy_instruct, b-mc2/sql-create-context, flytech/python-codes-25k, gbharti/finance-alpaca, medalpaca/medical_meadow_medical_flashcards
As an example, databricks/databricks-dolly-15k is a dataset that has different column names than the code expects.

On your PR description:
--output_dir /tmp/output\ -> --output_dir /tmp/output \

On your PR description: --output_dir /tmp/output\ -> --output_dir /tmp/output \

Fixed, thanks

PR was made and merged to the upstream repo


fix(add): Reduce the chance we re-format the user's [features] table

What does this PR try to resolve?
#11743 pointed out that we re-format the users [features] table when running cargo add  which was a bug introduced in #11099.
This reduces the chance people will run into this problem

Reducing the scope of the fmt call
Preserving formatting in a simple case

Actually removing the fmt case can make some common formatting cases more complex to do "right", so I'm punting on that for now.
How should we test and review this PR?
Look at the individual commits as I show how each change improves the behavior of cargo add.

@bors r+

:pushpin: Commit dbf9134d57354d222bbbbdd2539487470d5b4098 has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit dbf9134d57354d222bbbbdd2539487470d5b4098 with merge 8f49b5556bf8b6384fd3f26b91cd546ca527aba6...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing 8f49b5556bf8b6384fd3f26b91cd546ca527aba6 to master...


Support GPTQ/Marlin format quantization (4bit weight, f16 input)

GPTQ/Marlin quantization
Candle-vllm now supports GPTQ (Marlin kernel), you may supply the quant (marlin) and dtype (f16) parameters if you have Marlin format quantized models, such as:
cargo run --release -- --port 2000 --dtype f16 --weight-path /home/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4-Marlin/ llama3 --quant marlin

Tested speed: 115 tokens/s (batch size = 1), 753 tokens/s (batch size=16) for LLaMa3.1 8B. (almost double performance for single query compared to bf16 format)
You may use AutoGPTQ to transform a model to marlin format by loading the (quantized) model and supply the use_marlin=True in AutoGPTQ (which will generate marlin format quantized model once you call save_pretrained).
Note: only 4bit GPTQ quantization supported for marlin format at the moment, and the input data type should be f16 (--dtype f16). You need also renamed the transformed marlin format weight to "model.safetensors" and copy the "tokenizer.json" from the source model folder.
Further plan: in-situ convertion of any quantized models to marlin format for speeding up inference.

@guoqingbao nice work! 🤗


Improve XHorizontalTransformer

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 100.00%. Comparing base (8a10c61) to head (6df866e).
Report is 4 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #388   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           58        58
  Lines         1505      1509    +4
  Branches        59        59
=========================================
+ Hits          1505      1509    +4

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


[BrokenLinksH2] Fixed the link path

Fixed the link path

Docs Build status updates of commit 18fe5bf:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

biztalk/core/tools-and-utilities-to-use-for-troubleshooting.md
:white_check_mark:Succeeded
View

biztalk/install-and-config-guides/troubleshoot-known-issues-biztalk-install-setup.md
:white_check_mark:Succeeded
View

biztalk/technical-guides/best-practices-for-deploying-an-application.md
:white_check_mark:Succeeded
View

biztalk/technical-guides/sql-server-settings-that-should-not-be-changed.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 4c3ff65:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

biztalk/core/tools-and-utilities-to-use-for-troubleshooting.md
:white_check_mark:Succeeded
View

biztalk/install-and-config-guides/troubleshoot-known-issues-biztalk-install-setup.md
:white_check_mark:Succeeded
View

biztalk/technical-guides/best-practices-for-deploying-an-application.md
:white_check_mark:Succeeded
View

biztalk/technical-guides/sql-server-settings-that-should-not-be-changed.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit a6b4b42:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

biztalk/core/tools-and-utilities-to-use-for-troubleshooting.md
:white_check_mark:Succeeded
View

biztalk/install-and-config-guides/troubleshoot-known-issues-biztalk-install-setup.md
:white_check_mark:Succeeded
View

biztalk/technical-guides/best-practices-for-deploying-an-application.md
:white_check_mark:Succeeded
View

biztalk/technical-guides/sql-server-settings-that-should-not-be-changed.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel

Docs Build status updates of commit 3f5217c:
:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

biztalk/core/tools-and-utilities-to-use-for-troubleshooting.md
:white_check_mark:Succeeded
View

biztalk/install-and-config-guides/troubleshoot-known-issues-biztalk-install-setup.md
:white_check_mark:Succeeded
View

biztalk/technical-guides/best-practices-for-deploying-an-application.md
:white_check_mark:Succeeded
View

biztalk/technical-guides/sql-server-settings-that-should-not-be-changed.md
:white_check_mark:Succeeded
View

For more details, please refer to the build report.
Note: Broken links written as relative paths are included in the above build report. For broken links written as absolute paths or external URLs, see the broken link report.
For any questions, please:Try searching the docs.microsoft.com contributor guidesPost your question in the Docs support channel


CAT allocation API reports wrong disk usage

Describe the bug
While implementing searchable snapshot feature, we ran into search node disk exhaustion.
Trying to narrow it down, I noticed incorrect usage report by _cat/allocation API:
% curl logs:9200/_cat/allocation\?v\&s=host
shards disk.indices disk.used disk.avail disk.total disk.percent host
  2296        2.3tb     2.3tb        1tb      3.4tb           68 v80.co.com
  2297        2.3tb     2.3tb      1.1tb      3.4tb           68 v81.co.com
  2296        2.3tb     2.3tb        1tb      3.4tb           68 v82.co.com
  2296        2.4tb     2.4tb   1012.1gb      3.4tb           71 v83.co.com
   445      639.4gb    31.4gb     96.4gb    127.9gb           24 v87.co.com
As you can see, it reports that search node has 128GB disk and 31GB of it is in use.
However, the disk usage at the OS is quite different:
v87:[/opt/os-search]# df -h /mnt/search/
Filesystem      Size  Used Avail Use% Mounted on
/dev/xvdc1      128G  104G   25G  81% /mnt/search
I.e. 104GB is used out of 128GB, most of them (103GB) is the '/mnt/search/nodes' directory.
Related component
Other
To Reproduce

Install search node with dedicated data disk;
Restore some indices as "remote_snapshot";
Compare disk usage by '_cat/allocation' URL and by the OS 'df' output.

Expected behavior
'_cat/allocation' URL have to produce correct ouput.
Additional Details
Additional context
By the way, it seems that limiting cache size has no effect (maybe due to incorrect report?). Here is the current limit:
v87:[/opt/os-search]# grep cache config/opensearch.yml
node.search.cache.size: 102gb

Related issue: #11676


[Here's how To WatcH] Miami Dolphins vs Buffalo Bills Live Streams 𝚁eddiT?

Learn which TV channel or how to live stream the NFL Week 9 Buffalo Bills and Miami Dolphins game, Sunday, Nov. 3.
💻Best NFL Streams for PC/MAC 1080p/720p 👇👇
🔴Full HD:2024 NFL Football Live Free
📲Mobile friendly 1080p/720p 👇👇
🔴Mobile Free: NFL Week 9 Live HD
Upvote
1
Downvote
Share
Share
No-Deer-2719
OP
•
2h ago
•
More live games and exclusives ... Where to watch Dolphins vs. Bills: NFL kickoff time, TV channel, live stream, odds, prediction for AFC battle.
Upvote
1
Downvote
Share
Share
No-Deer-2719
OP
•
2h ago
•
If you're looking for an inexpensive way to watch a variety of football games, Fubo could be the best way to do so. The live TV streamer is ...
Upvote
1
Downvote
Share
Share
No-Deer-2719
OP
•
2h ago
•
Bills vs. Dolphins will air on CBS at 1 p.m. ET, and can be streamed live on DirecTV Stream (free trial), fuboTV (free trial) or Paramount+ ( ...


Doc update, including mid-December 2015 status

Update README.md with mid-December 2015 status
Update FAQ
Remove doc/overview.edoc, which had been checked in by accident (by me) and remained unnoticed
Change order of bullet points in doc/README.md

:+1:
LGTM


Add link check CI job

:white_check_mark: Checklist

[x] This pull request has a meaningful title.
[x] If your changes are not yet ready to merge, you have marked this pull request as a draft pull request.

:ballot_box_with_check: Maintainers' checklist

[x] This pull request has had the appropriate labels assigned
[x] This pull request has been added to the SATRE backlog project board
[x] This pull request has been assigned to one or more maintainers

:arrow_heading_up: Summary

:closed_umbrella: Related issues

Closes #38
:raising_hand: Acknowledging contributors

[ ] All contributors to this pull request are already named in the table of contributors in the README file.
[ ] The following people should be added to the table of contributors in the README file:

@manics Did you want to add your summary script? It looks useful. (I wonder if something that does the same is an upstream contribution Sphinx would be interested in)

@JimMadge Feel free to copy the script into this PR. I'm the only contributor to that script and I don't care about attribution 😄

I've merged in the main branch, fixed conflicts, and added the linkcheck script:


fix: install join nil point

What type of PR is this?

What does this PR do?

/kind bug
Which issue(s) does this PR fix?

Fixes #295
Special notes for your reviewer:

Does this PR introduce a user-facing change?

/lgtm
Good job！That's my fault，thx~ @rxy0210

/approve

/lgtm


Can't install the git-codereview command on mac, using contribution guide docs

Hello. I've tried installing git-codereview on my mac, using the provided command in Contribution Guide, but it doesn't work for me. Can somebody help me set it up?
I tried running go get -u golang.org/x/review/git-codereview
but can't get codereview to work.
I get this:
git: 'codereview' is not a git command. See 'git --help'.
Can somebody help me please?
What version of Go are you using (go version)?

go version go1.16.3 darwin/amd64

Does this issue reproduce with the latest release?
yes
What operating system and processor architecture are you using (go env)?
go env Output
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/paiman/Library/Caches/go-build"
GOENV="/Users/paiman/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/paiman/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/paiman/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.3"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/hs/6y9r9n0j75qfvv7gsb10kz6r0000gn/T/go-build2632970588=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?
I tried running:
go get -u golang.org/x/review/git-codereview
(even with a sudo)
What did you expect to see?
the help text
What did you see instead?
git: 'codereview' is not a git command. See 'git --help'.

is $GOPATH/bin in your $PATH?

is $GOPATH/bin in your $PATH?

But I have /usr/local/go/bin in my $PATH
Do I need to explicitly set $GOPATH for this to work?
(documentations seem incomplete IMHO)

For anyone having this same issue, Seems $GOPATH is the workspace directory.
these will help:
https://golang.org/doc/code#GOPATH

The contributing guide covers this:

Make sure git-codereview is installed in your shell path, so that the git command can find

and in general assumes you have a Go development environment setup.
Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.
For questions please refer to https://github.com/golang/go/wiki/Questions


set the default value for the ack_delay_exponent, if it is not sent

Codecov Report

Merging #1783 into master will increase coverage by 0.04%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master    #1783      +/-   ##
==========================================
+ Coverage   87.23%   87.27%   +0.04%
==========================================
  Files          97       97
  Lines        6405     6411       +6
==========================================
+ Hits         5587     5595       +8
+ Misses        586      584       -2
  Partials      232      232

Impacted Files
Coverage Δ

internal/handshake/transport_parameters.go
96.72% <100%> (+0.17%)
:arrow_up:

session.go
76.85% <0%> (+0.28%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7dc4be2...c7f1fce. Read the comment docs.


SeqLengthWarmup has no tests

We should have unit test coverage for SeqLengthWarmup.

Closing. Tracking elsewhere as low pri


Expire an invoice belonging to a sub accounts

Currently, the SDK doesn't allow passing sub-account ID for expiring an invoice even though API permits. This PR allows passing of optional subaccount ID parameter for Invoice Expiry.

@ErvanAdetya it's a problem with our upstream dependency compat with v1.11 & v1.12. Let's just merge this first. I will fix the CI later


[MM-27277] Enable stream mode for file uploads

Summary
PR enables stream mode for uploads by changing the order of fields in the multipart/form-data upload.
This enables the stream mode path which avoids a full memory allocation of the uploaded file.
Ticket
https://mattermost.atlassian.net/browse/MM-27277
Related PRs
https://github.com/mattermost/mattermost-server/pull/15616

This change is concerningly simple. I know there's server changes with it, but I didn't realize the order of these mattered.

Indeed and the server changes are actually unrelated to this specific case. The code to allow stream mode for multipart uploads was already there just needed this change which apparently was never done. The additional server PR does remove other allocations (yes, we were allocating more than once the same uploaded file).

@lindalumitchell
This code change is minimal and does not alter functionality.


Issue #3455: powershell_script: do not allow suppression of syntax errors

@ksubrama, @btm, @jdmundrawala  thanks for merging #3080. Here's an addendum based on a few pieces of feedback I didn't follow up on:

Took @jdmundrawala 's feedback that preserving behavior of other script resources that let you suppress syntax errors just added complexity at the expense of obscuring a much more likely scenario: user expects a (failure) return code of 1, but the script itself has a syntax error. In that case, the script needs to be fixed by the user, but the user isn't aware because Chef reports no error. The powershell_script resource hasn't been around long enough to need such strict backwards compatibility to justify the complexity in code (evidenced by numerous questions in the #3080 conversation) and the difficulty of the aforementioned scenario.
Feedback from @ksubrama and @jdmundrawala  around it not being obvious how we were actually doing the syntax check by embedding user code in a PowerShell script block. That's actually the core idea of the syntax validation, so to make that more obvious, I've restructured the code generation string so that instead of a terse one-liner, it's in multiple lines and looks closer to how you'd write idiomatic PowerShell code where the block was more obvious.

:+1:

:+1:


Devise and devise_token_auth omniauth callbacks

I'm using devise_token_auth version 0.1.30 with Devise 3.4.0
routes.rb looks like:
devise_for :users, controllers: { omniauth_callbacks: "omniauth_sessions", sessions: "sessions", registrations: "registrations", passwords: "passwords"}

namespace :api, defaults: { format: :json } do
    mount_devise_token_auth_for 'User', at: 'auth', controllers: { omniauth_callbacks: "omniauth_token_callbacks" }
...

Sign up and sign in paths for devise_token_auth works fine. But /api/auth/:provider/ returns
undefined method `valid_encoding?' for :json:Symbol

When I moved mount_devise_token_auth_for  outside api namespace an error occured:
ArgumentError: Invalid route name, already in use: 'new_user_session' as both Devise and devise_token_auth_for generate the same route new_user_session.
My main goal is to have different omniauth callbacks for Devise and token authentication (mobile app). I have two different controllers but I am unable to provide two different routes for them.

It is possible to use this gem alongside standard Devise.
What are the steps to reproduce the undefined method error?

It occurs when I am trying to access /api/:provider/

@lynndylanhurley Any insight on this issue?

@springerigor Did you find a solution?

Nope, it's obsolete so I'm closing this issue.


Migrate from macos-13 to macos-latest

Currently the .github/workflows/go-test-darwin.yaml uses the macos-13 runner.

https://github.com/senzing-garage/go-databasing/blob/6cdba7e64bbbbfa29f7439f90d14104012cd856b/.github/workflows/go-test-darwin.yaml#L30

This issue is to migrate to macos-latest,

Hint:
https://github.com/abiosoft/colima/issues/970#issuecomment-2298154164

Fixed via https://github.com/senzing-garage/go-databasing/issues/121


Support for h-feed from Microformats2

Implement a parser for h-feed, part of the microformats2 spec.
The format could be automatically discovered by looking for a h-feed or hfeed class in the HTML body.
Although this should be relatively easy to support, it is an old format that doesn't look like it is being widely used. Statistics on usage from both sites and readers, and how it compares to other already supported formats (such as RSS) should be taken in consideration before implementing it.

After doing a little bit of more research, microformats are easy to implement, but harder than I thought to parse. Mainly because microformats2 allows many different forms of structuring the HTML, which would require a lot of code to parse all scenarios. Also, microformats1 and microformats2 compatible at all, or at least not as compatible as RSS versions, which we would need two separate code bases for supporting both of them.
Due to the implementation complexity being higher than other feed types and microformats being used by very, very few websites, I don't see it being worth the time.
I'll close this for now, maybe in the future we can revisit the idea of implementing h-feed.


[BUG] require-hook and test.describe.configure

Top-level test.describe.configure gets flagged with this rule enabled, don't think this shouldn't happen

Also, I am not sure if it's by design, dotenv in playwright.config.ts is also getting flagged. Feels weird since it's not a test file and probably should be outside of scope of the plugin. Or is the expectation for this plugin is that it is only enabled for .spec. files?

Another example from the docs that gets flagged:
test.describe('A, runs in parallel with B', () => {
  test.describe.configure({ mode: 'default' });
  test('in order A1', async ({ page }) => {});
  test('in order A2', async ({ page }) => {});
});


Add a drop down box to limit returned features for the plot

Similar to pygeoapi, it would be useful to have a drop down box that limits how many features get loaded at once. Currently the line chart tries to load all data. For small collections this is fine but for ones with lots of data, this can be an issue and lag the UI.
If a country is using slow servers this would have bad UX.

Best approach would probably be to do the pagination for larger requests inside the UI rather than inside pygeoapi. You can even see the UI lag on large request just rendering all the data into plotly.
I will note that the primary use of the wis2box is not to store data persistently - I believe the US box only stores two days worth of history max. Relevant as we explore additional use cases for the wis2box-ui


AI configuration

Overview
Create and configure a OpenAI project. Research if an Agent will fit better in the project or if just a chat completion / model completion would be enough.

In the end, a chat completion functionality is best suitable for the quick exercise feature


ts_list to dataframe

Need a new functionality that transforms the legacy ts_list (from get_timeseries) into a data frame whose columns corresponds to the entries in the various dictionaries contained in the list.
Although this wouldn't replace a good querying function, it will enable users more familiar with pandas to work with the data directly.
@nickmckay Is this how LipdR works now?

Instead of creating a new function, add a parameter dataframe=True to return the data frame along the list.

done in 5907fa9a360d2d907216dd302a49e902be60c027


source/assets/js/main.js

Maybe I am a little confused, but should this file be in the repo? Isn't Mix creating it?
Same question about source/assets/mix-manifest.json

Right you are, I pulled out those unnecessary files. Thanks!


Fix incorrect reference to the "probe" key

The explanation text under lesson 8 oneliner incorrectly state that "name"
is the key for the full name of the probe but it should be "probe" as it
is used in the oneliner.

Thanks!