🚀 Feature: Dynamically disable options (try-it / authorize buttons) on Swagger page in API Catalog

🔖 Feature description
There are some APIs in the catalog that need to disable the try-it and the authorize buttons in the API Swagger page. I see this feature as a way to pass in the definitions as well as customizable plugins / options to the Swagger page (options and plugins used interchangeably going forward when talking about the SwaggerUI).
This would allow developers to send in options for APIs and have the Swagger page apply them. Note that it would be up to the developers passing in the options to correctly format options, but then it would allow for any of the Swagger UI options to be configurable based on the options passed. It should not affect APIs without the custom options, so it would only be an add on for APIs that require custom options.
🎤 Context
Some APIs in the catalog need to dynamically disable some of the Swagger configurable options via a plugin, specifically disabling the try-it button and the Authorize button. The Try-it button setups a test call and provides a sample response in the UI, and the Authorize button creates a sample authorization token for users to use when calling certain APIs.
Further extending the Swagger page to accept options / plugins for would allow for further extensibility and control over what APIs show. By making the Swagger page take options, any other customizable aspect of the Swagger UI would be configurable now at an API level.
✌ Possible Implementation
The Swagger UI will need to be passed plugin options in order to customize it:
<SwaggerUI spec={def} url="" plugins={plugins} deepLinking />
where the plugins are passed in through the props as seen here.
The plugins will also need to be propagated through this file.
Additionally, the options will need to be set and retrieved prior to being sent to the Swagger page, and I would imagine that having a predefined spec attribute would be a good way of doing this, and will have to be set prior to here.
Though I have not have a lot of time to really figure out how this might look, I would imagine that an API Processor could look for an Entity's spec.swaggerPlugins (or something of that sort) and transform it into what the SwaggerUI could process. Then, in addition to passing the definition to the SwaggerUI, it could also pass the swaggerPlugins.
By propagating the options from an API, the Swagger page can dynamically render the Swagger pages differently for different APIs. It is important to make sure that the APIs without any options are not affected, but by processing the Entity's options prior to passing the options to the SwaggerUI element, there should not be any negative effect on APIs that don't pass options.
👀 Have you spent some time to check if this feature request has been raised before?

[X] I checked and didn't find similar issue

🏢 Have you read the Code of Conduct?

[X] I have read the Code of Conduct

Are you willing to submit PR?
Yes I am willing to submit a PR!

Hi! I think this make sense to be able to do that. Feel free to submit a proposal as a PR 👍

Hey @jhaals,
I am running into some typing errors due to the fact that all spec attributes need to be a JsonObject which will recursively simplify down to all primitive types. The plugins will likely need to be more complex, including a lamda function as a valid attribute of the JsonObject: () => () => false. One way to bypass this using the current approach is to store the plugins as a string until right before the SwaggerUI gets it (and convert it using something like JSON.parse(plugins_string_representation)), but I don't think that is a great way to go about this. Do you have any recommendations for where else this information could be stored at the entity level so it would not have to be translated into a string and back again (or any other ideas under the current constraints)?
Thanks!

@whitemanthedj you won't be able to store anything else than JsonObject in the catalog, that's a constraint to work with x)
I think a reasonably light-weight way forward for this is to add one or more annotations that are known by the API docs plugin, that are read and used to configure the UI.

If I understand correctly, then that will lose the generic ability for any of the Swagger plugins. If apis need additional plugins in the future that I don't codify now for the swagger page, then they would have to update this repo, is that what you meant?

@whitemanthedj kind of, you could also add some customization points that makes it possible for users of the API docs plugin to customize the logic. That way you may not need to change code in this repo, but your own repo.

For future readers: see closing comments in #16169. It may be best to not so much focus on explicit customizability of the api-docs plugin's top level components themselves (such as EntityApiDefinitionCard/ApiDefinitionCard). Instead, the base idea should probably be that users supply entirely custom views (by means of the apiDocsConfigRef). Then, any work related to the ask in here, might want to focus on what can be done to make it less copy-paste-heavy to make such custom views. Are there any helper components that need to be created or exported, for example? Then the onus on passing in plugins, or giving custom props, is put entirely on the adopter without having to go through the layers of the framework as such.


Better testing for Monge-Ampere boundary conditions

Closes #79.
Partially addresses #19.
This PR achieves 100% code coverage for the monge_ampere module. In particular, it introduces unit tests for the boundary condition logic for the equidistribution equation. The boundary condition code is refactored, including pulling it out as a separate method so it can be more easily targeted.
The PR also introduces a new math module including functions for generating equations of lines and planes. These are used to verify that boundary conditions are only applied on straight line boundary segments (2D) and planar boundary segments (3D). The new module has 100% code coverage.
Finally, an additional error check test is added for the tangling module to get that to 100% code coverage, too.

Putting down everyone as reviewer in case they have time to look at this. Would be good to get it merged so we can carry on with other MA changes. Feel free to only leave a partial review with comments on certain parts if you like.

Thanks @jwallwork23! I have yet to dive deep into the underlying movement code so I will leave it to others to review for now :) otherwise I will get into it in the next 2-3 weeks

@stephankramer Do you have time to review this by any chance? Would be much appreciated.

Note that #115 will need merging into the branch associated with this PR before we merge this one.


Replaced tag always, which colidate when including role.

Renamed tags to include service name, for better handling with other roles.
This solves #101 by which I was affected as well.
Additionally tag naming will help, to ie. configure only grafana, not also prometheus, if both roles are included in your task.

Thanks!


Development of the Petrolhead backend

The first step to creating a cross-platform version of Petrolhead is getting the Azure Mobile Apps backend completed and online. This requires the following steps:

[ ] Completion of data entities
[ ] Addition of support for $expand, which is necessary to have relationships between different entities.
[ ] Added push notification functionality

@SupernovaApps/petrolhead-community, can you help out? I'll have the initial infrastructure in the next day or so, I'm quite busy at the moment with other stuff.

Hey all,
I've run into a little snag while developing the backend. I'm not entirely sure how to integrate support for Entity Framework relationships, which are necessary for Petrolhead to function with Azure Mobile Apps.
@JerryNixon & @darenm, do you have any advice on how to do this? I'm currently looking at this layout:
Vehicle - Expenses (1:n)
Vehicle - Reminders (1:n)
Do you have any advice (or better yet, a pull request)?
@SupernovaApps/petrolhead-community, I'd be grateful for your help too.

The backend is functional - I still need to make one or two changes, but all the functionality works as expected. I'll commit the completed changes in a few minutes, they are already live on the server.
@SupernovaApps/petrolhead-community, you can now start working on implementing the client apps! :+1:

The necessary changes have now been made. The latest code now supports authentication, and I updated the backend to filter results based on your user ID. Could @SupernovaApps/petrolhead-community members test that this functionality is fully working?


gparyani: Watch rdos.net

gparyani requests the watch of the watch_keyword rdos\.net. See the MS search here and the Stack Exchange search here.

rdos\.net has been seen in 0 true positives, 0 false positives, and 0 NAAs.

Coverage remained the same at 64.618% when pulling fcb96967e6c9b0cd45a44437d6679c7bfda001e4 on auto-blacklist-1539680588 into e25e082bc4cb4fc60744305a3894bfe252827b29 on master.


Changes to Travis CI pricing

It should be checked, what the impact of this new situation is:
https://blog.travis-ci.com/2020-11-02-travis-ci-new-billing

changes for move from Travis CI to github actions have been submitted.


feat(esm): support custom conditions

Add support for customConditions in ESM.
For example, if a tsconfig contains this:
{
	"compilerOptions": {
		"customConditions": ["source"],
	}
}

passing the condition as an arg to the CLI would not be needed anymore:
- tsx --conditions source src
+ tsx src

But this is only for ESM as explained here: https://github.com/privatenumber/tsx/issues/574#issuecomment-2156442343
Related to #574

Thanks for the PR
This is blocked until we add condition support to the CJS resolver though. Since tsx aims to consolidate ESM and CJS, we want to avoid introducing discrepancies between them.


2427 fix spearbit pr 34

Description

Type of change

[ ] Docs change / dependency upgrade
[ ] Configuration / tooling changes
[ ] Refactoring
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Requires changes in customer code

High-level change(s) description - from the user's perspective

Related Issue(s)
Fixes
Related pull request(s)

Updated and fixed.

Merged into 2152 in #2532


Contrast Issues on About Page

The suggested citation on the About page has too low of a contrast - found using WAVE on a Mac.
Screenshots attached.

duplicate of #27


Can I add custom response after login?

Can I add custom response?
Currently After login
{
    "key": "4cada2ed9ccdfa78313fcfba075bf87ecb9d05b50"
}

Getting above response
But If I want to add more parameters like
{
    "key": "4cada2ed9ccdfa78313fcfba075bf87ecb9d05b5",
    "User" {
        "name": "XXCD",
        "age": 20
    }
}

Yes, you can create a CustomLoginView subclassing the LoginView provided by rest_auth, but I recommend you use djangorestframework jwt authentication


Look into why manifest generation is breaking

Describe the bug
DCA is currently not able to generate manifest templates.

This has been tested with both empty folders and folders that contain files.
To Reproduce
Steps to reproduce the behavior:

Go to DCA > Get Metadata Template
Select any Project, Folder, and Template from the top selection bar
Click on the button to generate a Google Sheets template
See error (semi-hidden under the Waiter window)

Expected behavior
A Google Sheets should've generated.
Priority (select one)

[ ] Minor ⬇
[ ] Major 📢
[x] Critical 🆘

Tagging @afwillia @rrchai @milen-sage for possible guidance/assistance.

I attempted to look more into this issue but the logs aren't updating so I don't know where to start.  @afwillia @rrchai @milen-sage do you guys have an idea as to possible reasons why this is failing?
Also tagging @ychae for awareness.

@vpchung The logs of app will be updated in https://www.shinyapps.io/admin/#/application/5992584/logs and the login info can be found in lastpass. It might need to wait a minute for the logs to be updated after your login.
The error of this behavior is here:
2022-06-14T03:46:00.295432+00:00 shinyapps[5992584]: Warning: Error in py_call_impl: RefreshError: ('invalid_grant: Token has been expired or revoked.', {'error': 'invalid_grant', 'error_description': 'Token has been expired or revoked.'})
2022-06-14T03:46:00.295481+00:00 shinyapps[5992584]:
2022-06-14T03:46:00.295534+00:00 shinyapps[5992584]: Detailed traceback:
2022-06-14T03:46:00.295637+00:00 shinyapps[5992584]:     mg = ManifestGenerator(
2022-06-14T03:46:00.295596+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/schematic/models/metadata.py", line 141, in getModelManifest
2022-06-14T03:46:00.295694+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/schematic/manifest/generator.py", line 43, in __init__
2022-06-14T03:46:00.295738+00:00 shinyapps[5992584]:     services_creds = build_credentials()
2022-06-14T03:46:00.295786+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/schematic/utils/google_api_utils.py", line 53, in build_credentials
2022-06-14T03:46:00.295823+00:00 shinyapps[5992584]:     creds = generate_token()
2022-06-14T03:46:00.295861+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/schematic/utils/google_api_utils.py", line 40, in generate_token
2022-06-14T03:46:00.295908+00:00 shinyapps[5992584]:     creds.refresh(Request())
2022-06-14T03:46:00.296014+00:00 shinyapps[5992584]:   [No stack trace available]
2022-06-14T03:46:00.295963+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-pack [... truncated]

It seems to do something with invalid google api credentials that cause backend function's failure. I am not sure if it is just because of invalid token.pickle or other credential files (credentials.json, service_account_creds.json) stored in Github Secrets. I also tag @mialy-defelice and @GiaJordan who could have more insights about this error.

@vpchung I think the token.pickle secret needs to be updated. I will do that now and rerun the deployment action.

Thank you both!
But yeah, the logs weren't updating on my end - I looked at the same link you provided.  Is there a way to trigger the log updates?

@vpchung  On my end, every time I start the app, I would need to refresh the log page and the log will be updated. If you go the log page now,  can you see the logs of my login and error:
2022-06-14T17:02:14.300258+00:00 shinyapps[5992584]: INFO: [2022-06-14 17:02:13] synapseclient_default - Welcome, Rongrong Chai!
2022-06-14T17:02:14.300211+00:00 shinyapps[5992584]:
2022-06-14T17:02:14.300302+00:00 shinyapps[5992584]:
2022-06-14T17:02:14.299967+00:00 shinyapps[5992584]:     pip install --upgrade synapseclient
2022-06-14T17:02:14.299410+00:00 shinyapps[5992584]:     pip install --upgrade synapseclient
2022-06-14T17:02:14.300346+00:00 shinyapps[5992584]:
2022-06-14T17:02:14.299460+00:00 shinyapps[5992584]:
2022-06-14T17:02:14.299504+00:00 shinyapps[5992584]: Python Synapse Client version 2.6.0 release notes
2022-06-14T17:02:14.299636+00:00 shinyapps[5992584]:
2022-06-14T17:02:14.299547+00:00 shinyapps[5992584]:
2022-06-14T17:02:14.300077+00:00 shinyapps[5992584]: Python Synapse Client version 2.6.0 release notes
2022-06-14T17:02:14.300032+00:00 shinyapps[5992584]:
2022-06-14T17:02:14.299897+00:00 shinyapps[5992584]: A more recent version of the Synapse Client (2.6.0) is available. Your version (2.5.1) can be upgraded by typing:
2022-06-14T17:02:14.300389+00:00 shinyapps[5992584]: Listening on http://<IP_ADDRESS>:35871
2022-06-14T17:02:14.300121+00:00 shinyapps[5992584]:
2022-06-14T17:02:14.299678+00:00 shinyapps[5992584]: Welcome, Rongrong Chai!
2022-06-14T17:02:30.299164+00:00 shinyapps[5992584]: Warning: Error in py_call_impl: KeyError: 'NA'
2022-06-14T17:02:30.299229+00:00 shinyapps[5992584]:
2022-06-14T17:02:30.299296+00:00 shinyapps[5992584]: Detailed traceback:
2022-06-14T17:02:30.299364+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/schematic/models/metadata.py", line 141, in getModelManifest
2022-06-14T17:02:30.299411+00:00 shinyapps[5992584]:     mg = ManifestGenerator(
2022-06-14T17:02:30.299469+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/schematic/manifest/generator.py", line 87, in __init__
2022-06-14T17:02:30.299603+00:00 shinyapps[5992584]:     required_dependencies = self.se.explore_class(source_node)["dependencies"]
2022-06-14T17:02:30.299512+00:00 shinyapps[5992584]:     is_file_based = "Filename" in self.sg.get_node_dependencies(self.root)
2022-06-14T17:02:30.299558+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/schematic/schemas/generator.py", line 210, in get_node_dependencies
2022-06-14T17:02:30.299653+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/schematic/schemas/explorer.py", line 332, in explore_class
2022-06-14T17:02:30.299801+00:00 shinyapps[5992584]:   [No stack trace available]
2022-06-14T17:02:30.299701+00:00 shinyapps[5992584]:     if "subClassOf" in self.schema_nx.nodes[schema_class]:
2022-06-14T17:02:30.299754+00:00 shinyapps[5992584]:   File "/srv/connect/apps/csbc_data_curator_staging/.venv/lib/python3.8/site-packages/networkx/classes/reportview [... truncated]

You could also view see all the logs after the app was initiated by downloading it.

@rrchai I don't use DCA and don't have access to those logs. Would you be able to paste the entire stack here? I just created an issue here for an error I noticed in generating manifests. Im unsure if this is related. It looks different.
@vpchung could you provide me your Data Model and the name of the manifest you were attempting to generate?

@rrchai okay, downloading the logs triggered it - thanks!
@mialy-defelice thank you for extending the help!  Luckily, our issue was a simple secrets thing, which @afwillia has fixed.  That being said, only the Consortium template is being generated, despite clicking on other types.... could that be related to the issue you filed?
Regardless, closing this ticket now.  Thanks, everyone!

@vpchung No that should not be related to the other issue. Could you file a ticket for that issue? I have not seen anything on the backend like that but I could check what it does when working through command line. If its fine through the CLI we could move the ticket to the DCA in case its a front end issue. I would just need your model and the names of the other templates you are attempting to make.

@mialy-defelice I was able to figure out that the bug is frontend related, so I'll file it under the DCA.  Thanks!


Changes to slim generation

[ ] Ensure versionIRI/data-version present for both obo and owl
[ ] Ensure purls resolve - use new style versionIRI format with /go/releases/
[ ] Only use limited relationship set in slims, remove has_part

still valid?

Can move to #14276

looks like some slims still have has_part so leave open for now

I think this is no longer relevant


Enable GENCODE annotations

I'll write more detailed issue at a later time. But briefly...

Implement some form of lazy loading: #109 or #111
Implement a transform that splits GTF's key-value column into objects. Or alternatively, a GTF3 loader could do it.
Implement support for more complex data flows. Currently, each flow node has a maximum of one parent and each data flow has only a single data source.
Implement a "lookup" transform that joins data from two flow nodes using a key. Something similar to: https://vega.github.io/vega-lite/docs/lookup.html

The required data flow looks roughly like this:

Filter the annotation and retain transcripts.

Compute a layout using the "pileup" transformation
Publish it as piledTranscripts
Display using "rule" mark

Filter the annotation and retain exons, UTRs, etc.

Join the data flow to piledTranscripts using transcript_id
Display using a suitable mark, probably "rect"

I think I'll use GMOD's gff-js once this issue has been resolved: https://github.com/GMOD/gff-js/issues/62
Making the current version work with Vite seems difficult.

Closed by #177


fix up links in readme

There were several broken and/or irrelevant links since the launch of docs.datafold.com 3.0, as well as an outdated Loom.

I think there is another open PR for some of this

this? https://github.com/datafold/data-diff/pull/471/files

@dlawin Yes.
@leoebfolsom Now that I am looking at it, I think we can split that one in half - the README updates and the Python API doc updates. The README updates can be added to this one


FFMPEG process pileup

This could be a question or a bug; I'm honestly not sure. Basically, through the course of my music bot running, it generally accumulates large sums of FFMPEG processes that never die and use up a fair bit of RAM. AFAIK, I am stopping the audio correctly. Any idea why this might occur?

I seem to be having the same issue with imortal FFMPEG processes.

Not sure how to fix this issue. I tried modifying AudioEncoder to terminate the FFMPEG encoding process when it finishes. However, processes still end up not being terminated. Can't seem to figure out why.

Should be solved by #390! :D


Pytorch Lightning deprecated "gpus" argument in trainer

Pytorch lightning will remove the "gpus" argument in the trainer in favour of using "accelerator" and "devices". See docs:
https://pytorch-lightning.readthedocs.io/en/stable/api/pytorch_lightning.trainer.trainer.Trainer.html?highlight=trainer

This now causes an error when trying to run the quick start tutorial. How can we fix this?
TypeError: __init__() got an unexpected keyword argument 'gpus'

Thanks @ochalkley-advai for raising this. We're working to fix it now!

Thanks @thomasgaudelet, I tried PyTorch 1.13 but that didn't work strangely. I did find a hacky workaround of commenting out the gpus arg in the pl.Trainer call on line 83 of pyrelational/model_managers/lightning_model_manager.py and adding accelerator="auto",. This should do for the time being but I would definitely like a proper fix ASAP. I don't have experience contributing to open-source projects but happy to help out on this if there is anything I can do?

Have you tried downgrading pytorch_lighting to 1.8.6 for instance?

Addressed in PR #112 on main branch, planning for next release by end of this week.


Enhancement: Add property chromeDriverBinary to enable specification in wdio.conf.js

Hello
I am not sure how to use a self-prescribed chromedriver binary which may be required by a software development kit.
I don't know how to specify a chromedriver binary of your choice in the local file system.
Even if I were attempting to use the chromedriver in wdio-chromedriver-service that is supposed to work "out of the box", I am not sure I know how to make that work that either. I followed the instructions but unable to avoid: Error: Couldn't initialise "wdio-chromedriver-service".
With the same wdioconf.js file (as the one that produces the output below) I can comment out one line:
services: ['chromedriver']
then manually start the SDK-provided chromedriver, from another shell: ./bin/chromedriver --port=9515, then run wdio. That works as expected.
I would like to know how to do two things:

Specify the path to a special chromedriver binary that exists in my file system
Get the wdio-chromedriver-service to use the chromedriver included as a peer dependency

I read the other issues to try to find the answer. Particularly notable is #11. It was a surprise to hear @christian-bromann not imagine a reason to use a specific older version of software as opposed to the latest and current version. Maybe he is only thinking about things in the realm of being "out in front" of the latest browser changes and not the creation of test cases for legacy code.
Thank you for writing this service. It is just the right functionality to encapsulate! I appreciate your foresight.
If any resulting code or documentation changes result from my question, then I will watch closely with intent of helping to support this package.
And please let me know if the log is insufficient and does not give you everything you may need to help me.
$ DEBUG=true ./node_modules/.bin/wdio wdio.conf.js
2019-05-19T22:53:05.132Z DEBUG @wdio/utils:initialiseServices: initialise wdio service "chromedriver"
2019-05-19T22:53:05.145Z ERROR @wdio/utils:initialiseServices: Error: Couldn't initialise "wdio-chromedriver-service".
Error: Cannot find module 'chromedriver'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:580:15)
    at Function.Module._load (internal/modules/cjs/loader.js:506:25)
    at Module.require (internal/modules/cjs/loader.js:636:17)
    at require (internal/modules/cjs/helpers.js:20:18)
    at Object.<anonymous> (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/wdio-chromedriver-service/build/launcher.js:10:44)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
    at safeRequire (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/utils/build/initialisePlugin.js:19:13)
    at initialisePlugin (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/utils/build/initialisePlugin.js:56:18)
    at initialiseServices (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/utils/build/initialiseServices.js:64:53)
    at Launcher.run (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/cli/build/launcher.js:70:52)
    at launch (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/cli/build/run.js:88:12)
    at run (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/cli/build/run.js:64:12)
    at Object.run (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/cli/build/index.js:45:20)
    at Object.<anonymous> (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/cli/bin/wdio.js:11:21)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
2019-05-19T22:53:05.146Z INFO @wdio/cli:Launcher: Run onPrepare hook
2019-05-19T22:53:05.148Z INFO @wdio/local-runner: Start worker 0-0 with arg: wdio.conf.js

RUNNING  0-0 in chrome - /test/specs/basic.js

Stdout:
[0-0] 2019-05-19T22:53:05.339Z INFO @wdio/local-runner: Run worker command: run
[0-0] 2019-05-19T22:53:05.343Z DEBUG @wdio/utils:initialiseServices: initialise wdio service "chromedriver"
[0-0] 2019-05-19T22:53:05.360Z DEBUG @wdio/local-runner:utils: init remote session
[0-0] 2019-05-19T22:53:05.382Z INFO webdriver: [POST] http://<IP_ADDRESS>:9515/session
[0-0] 2019-05-19T22:53:05.382Z INFO webdriver: DATA { capabilities:
    { alwaysMatch: { browserName: 'chrome', chromeOptions: [Object] },
      firstMatch: [ {} ] },
   desiredCapabilities:
    { browserName: 'chrome',
      chromeOptions:
       { binary:
          '../../desktop/sdk/osx32/chromium.app/Contents/MacOS/chromebrowser' } } }
[0-0] 2019-05-19T22:53:05.392Z INFO webdriver: Retrying 1/3
[0-0] 2019-05-19T22:53:05.392Z INFO webdriver: [POST] http://<IP_ADDRESS>:9515/session
[0-0] 2019-05-19T22:53:05.392Z INFO webdriver: DATA { capabilities:
    { alwaysMatch: { browserName: 'chrome', chromeOptions: [Object] },
      firstMatch: [ {} ] },
   desiredCapabilities:
    { browserName: 'chrome',
      chromeOptions:
       { binary:
          '../../desktop/sdk/osx32/chromium.app/Contents/MacOS/chromebrowser' } } }
[0-0] 2019-05-19T22:53:05.393Z INFO webdriver: Retrying 2/3
[0-0] 2019-05-19T22:53:05.394Z INFO webdriver: [POST] http://<IP_ADDRESS>:9515/session
[0-0] 2019-05-19T22:53:05.394Z INFO webdriver: DATA { capabilities:
    { alwaysMatch: { browserName: 'chrome', chromeOptions: [Object] },
      firstMatch: [ {} ] },
   desiredCapabilities:
    { browserName: 'chrome',
Stdout:
[0-0] 2019-05-19T22:53:05.339Z INFO @wdio/local-runner: Run worker command: run
[0-0] 2019-05-19T22:53:05.343Z DEBUG @wdio/utils:initialiseServices: initialise wdio service "chromedriver"
[0-0] 2019-05-19T22:53:05.360Z DEBUG @wdio/local-runner:utils: init remote session
[0-0] 2019-05-19T22:53:05.382Z INFO webdriver: [POST] http://<IP_ADDRESS>:9515/session
[0-0] 2019-05-19T22:53:05.382Z INFO webdriver: DATA { capabilities:
    { alwaysMatch: { browserName: 'chrome', chromeOptions: [Object] },
      firstMatch: [ {} ] },
   desiredCapabilities:
    { browserName: 'chrome',
      chromeOptions:
       { binary:
          '../../desktop/sdk/osx32/chromium.app/Contents/MacOS/chromebrowser' } } }
[0-0] 2019-05-19T22:53:05.392Z INFO webdriver: Retrying 1/3
[0-0] 2019-05-19T22:53:05.392Z INFO webdriver: [POST] http://<IP_ADDRESS>:9515/session
[0-0] 2019-05-19T22:53:05.392Z INFO webdriver: DATA { capabilities:
    { alwaysMatch: { browserName: 'chrome', chromeOptions: [Object] },
      firstMatch: [ {} ] },
   desiredCapabilities:
    { browserName: 'chrome',
      chromeOptions:
       { binary:
          '../../desktop/sdk/osx32/chromium.app/Contents/MacOS/chromebrowser' } } }
[0-0] 2019-05-19T22:53:05.393Z INFO webdriver: Retrying 2/3
[0-0] 2019-05-19T22:53:05.394Z INFO webdriver: [POST] http://<IP_ADDRESS>:9515/session
[0-0] 2019-05-19T22:53:05.394Z INFO webdriver: DATA { capabilities:
    { alwaysMatch: { browserName: 'chrome', chromeOptions: [Object] },
      firstMatch: [ {} ] },
   desiredCapabilities:
    { browserName: 'chrome',
      chromeOptions:
       { binary:
          '../../desktop/sdk/osx32/chromium.app/Contents/MacOS/chromebrowser' } } }
[0-0] 2019-05-19T22:53:05.394Z INFO webdriver: Retrying 3/3
[0-0] 2019-05-19T22:53:05.394Z INFO webdriver: [POST] http://<IP_ADDRESS>:9515/session
[0-0] 2019-05-19T22:53:05.395Z INFO webdriver: DATA { capabilities:
    { alwaysMatch: { browserName: 'chrome', chromeOptions: [Object] },
      firstMatch: [ {} ] },
   desiredCapabilities:
    { browserName: 'chrome',
      chromeOptions:
       { binary:
          '../../desktop/sdk/osx32/chromium.app/Contents/MacOS/chromebrowser' } } }

Stderr:
[0-0] 2019-05-19T22:53:05.358Z ERROR @wdio/utils:initialiseServices: Error: Couldn't initialise "wdio-chromedriver-service".
Error: Cannot find module 'chromedriver'
     at Function.Module._resolveFilename (internal/modules/cjs/loader.js:580:15)
     at Function.Module._load (internal/modules/cjs/loader.js:506:25)
     at Module.require (internal/modules/cjs/loader.js:636:17)
     at require (internal/modules/cjs/helpers.js:20:18)
     at Object.<anonymous> (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/wdio-chromedriver-service/build/launcher.js:10:44)
     at Module._compile (internal/modules/cjs/loader.js:688:30)
     at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
     at Module.load (internal/modules/cjs/loader.js:598:32)
     at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
     at Function.Module._load (internal/modules/cjs/loader.js:529:3)
     at safeRequire (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/utils/build/initialisePlugin.js:19:13)
     at initialisePlugin (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/utils/build/initialisePlugin.js:56:18)
     at initialiseServices (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/utils/build/initialiseServices.js:64:53)
     at Runner.run (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/runner/build/index.js:88:35)
     at process.on.m (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/local-runner/build/run.js:37:20)
     at process.emit (events.js:187:15)
     at emit (internal/child_process.js:812:12)
     at process._tickCallback (internal/process/next_tick.js:63:19)
[0-0] 2019-05-19T22:53:05.392Z WARN webdriver: Request failed due to connect ECONNREFUSED <IP_ADDRESS>:9515
[0-0] 2019-05-19T22:53:05.393Z WARN webdriver: Request failed due to connect ECONNREFUSED <IP_ADDRESS>:9515
[0-0] 2019-05-19T22:53:05.394Z WARN webdriver: Request failed due to connect ECONNREFUSED <IP_ADDRESS>:9515
[0-0] 2019-05-19T22:53:05.395Z ERROR webdriver: Request failed due to Error: connect ECONNREFUSED <IP_ADDRESS>:9515
     at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1113:14)
[0-0] 2019-05-19T22:53:05.395Z ERROR @wdio/runner: Error: connect ECONNREFUSED <IP_ADDRESS>:9515
     at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1113:14)
[0-0] 2019-05-19T22:53:05.396Z ERROR @wdio/local-runner: Failed launching test session: TypeError: Cannot read property 'capabilities' of null
     at Runner.run (/Users/tester/go/src/github.com/acme/webapps/app1/integration/webdriverio/node_modules/@wdio/runner/build/index.js:91:74)
     at process._tickCallback (internal/process/next_tick.js:68:7)

Worker Error:
Error: connect ECONNREFUSED <IP_ADDRESS>:9515
     at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1113:14)

Test Suites:	 0 passed, 1 failed, 1 total (100% completed)
Time:		 🕔  0.28s

error Command failed with exit code 1.

Thank you very much!

And please let me know if the log is insufficient and does not give you everything you may need to help me.

Please see install instructions, you need to install chromedriver too via:
$ npm install chromedriver --save-dev

I read the other issues to try to find the answer. Particularly notable is #11. It was a surprise to hear @christian-bromann not imagine a reason to use a specific older version of software as opposed to the latest and current version. Maybe he is only thinking about things in the realm of being "out in front" of the latest browser changes and not the creation of test cases for legacy code.

Not sure to which of my comment you are referring to but imo there is not much reason to test against an older version of Chrome as the browser updates quite regularly and updates happen automatically. But sure, every use case is different and there is nothing wrong doing that if your requirements say so.

Thank you for your input Christian. I appreciate your great work. I am testing legacy internal apps on 32 bit chromium browser. Based on my attempts I discover modern chromedriver binaries would not interoperate with the 32-bit run-time world in which I test.
It would be convenient and intuitive if we could specify the filepath of a particular chromedriver binary in the wdio.conf.js.
The wdio-chromedriver service is a great abstraction point.
wdio-chromedriver service has facility for chromeDriverArgs and chromeDriverLogs.
Does it have somewhere, chromeDriverBinary?
If not, that seems a reasonable knob to bring to level of the wdio.conf.
Thanks for your guidance, Christian. You are doing amazing things with webdriverio.

I am beginning to see some "usability" issues surrounding these packages.
I am happy to determine where they go awry and fix them...
Can I run the node-chromedriver installer and set breakpoints on a cloned copy of it in my filesystem?
The node-chromedriver package may have the usability problem.
Apparently, one cannot install node-chromedriver and then configure it afterwards (and before run-time) to use a specific chromedriver binary.
Once installed, node-chromedriver appears to lock you into the chromedriver that was specified at install time. At least that is what I think at this point.
When I try to specify CHROMEDRIVER_FILEPATH at install time I do not get the result I expected.

Hi, I'm guessing here, but what if you install a specific version of https://www.npmjs.com/package/chromedriver?activeTab=versions which is pinned to the version of Chrome you want to use for your tests, i.e. npm install<EMAIL_ADDRESS>Also, according to http://chromedriver.chromium.org/capabilities you should be able to specify the exact binary with chromeOptions.binary-option.
Either way, Chrome 32bit for Mac seems to have been dropped in 2014 according to https://gsuiteupdates.googleblog.com/2014/09/google-chrome-64-bit-for-mac-and-windows.html, and you cannot expect new tools to support these things. Maybe Browserstack has support for your os/version of Chrome? If so, use https://webdriver.io/docs/browserstack-service.html.

I appreciate that ideas. Very informative.
I prefer not to use the older topology such as selenium when the chromedriver itself speaks the same "language" or wire protocol as it seems to be called.
A quick solution that works is to use the default installation of node-chromedriver and hack in your favorite chromedriver:
Copy the chromedriver one wants to use, to the filepath: ./node_modules/chromedriver/lib/chromedriver/chromedriver
The interoperability between packages [webdriverio, wdio-chromedriver-service, node-chromedriver] appears to work. But it is unfortunate that the package, node-chromedriver, on which wdio-chromedriver-service depends, appears to have a broken installer when trying to use the custom binary feature (FWICT).

@atti187 Your service is very useful. Thanks for writing it.


[discussion] how to show content in menu in front-end but not backend?

Laravel Version: 5.3
Voyager Version: v0.10.4
PHP Version: 7.0
Database Driver & Version: MySQL 5.7.11

Description:
I tried use php artisan make:auth
and insert <?=Menu::display('main', 'bootstrap') ?> in front-end views\layouts\app.blade.php
Add BREAD to table "products" and use MenuBuilder
And the main menu can display in front-end successfully.
But when click button in menu, it will turn into backend display /admin/products
Is there some methods which voyager design for us to display the table in the front-end?
Or what should I do for customizing this?
Thanks a lot.
Steps To Reproduce:

If I understand you correctly, you wish to add a menu item with the link to view all your products.
Then what I would do is to make a menu item with a link to /products.
Then I would make a route for that like this:
Route::get('/products', function () {
    $products = App\Products::all();
    return view('products.list', [
        'products' => $products,
    ]);
});

Then you can show your products there to the front-end users.

NOTE: It's on my todo to make some helpers and guides for using Voyager on the front-end

Sorry I do not mention it clearly.
My question is :
Can I use the BREAD function in the frontend page (http://localhost/home/ ) ?
Which do not need admin permission like these
"
X Admin
X Browse Admin
X Browse Database
X Browse Media
X Browse Settings
"
just need these permission and URL not in "localhost/admin/product"
"
V Product
V Browse Product
V Read Product
V Edit Product
V Add Product
V Delete Product
"
Thanks a lot.

You can check the user permission using Voyager::can('browse_product')

@akpotter, Voyager is intended to be an Admin tool, not frontend functionality.  You can build Menus and display them in the front-end, but the BREAD functionality is specific to Voyager.  If you need front-end functionality to let end-users create/edit things, then you will need to create forms and use the model that Voyager created.
Apologies if I misunderstood your question.  If that's the case, please clarify and let me know.

Got it! Nice job! I will try do this frontend functionality by my self. Thanks for all your contribution.
And These is a remote code execution vulnerability in Voyager right now. Later someday I will create an issue for it.

@akpotter, please send me a email with the security vulnerability please.
<EMAIL_ADDRESS>

Hi,
Laravel 5.5 / Voyager 1.0
Can we hide some views to use who are not admin? for exemple some pages or products which are not published and want to let offline.
Thanks

Hey @decomstore
Please create your own issue for this, since this have been closed for long time.
NOTE: You can read how to do such thing using the policies here: https://webman.io/blog/post/using-policies-in-laravel-voyager


Azure CLI 2.43.0 bug

az feedback auto-generates most of the information requested below, as of CLI version 2.0.62

azure-cli                         2.43.0
core                              2.43.0
telemetry                          1.0.8
Dependencies:
msal                              1.20.0
azure-mgmt-resource             21.1.0b1
Related command
az monitor diagnostic-settings create
Describe the bug
az monitor diagnostic-settings create if you run this command with this version, it is failed with below message, same command perfectly works with previous version.
Exception
ERROR: The command failed with an unexpected error. Here is the traceback:
ERROR: 'AAZCommandCtx' object has no attribute 'resource'
Traceback (most recent call last):
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\knack/cli.py", line 233, in invoke
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 663, in execute
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/commands/init.py", line 697, in _run_job
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/core/aaz/_command.py", line 145, in call
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/monitor/aaz/latest/monitor/diagnostic_settings/_create.py", line 33, in _handler
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/monitor/aaz/latest/monitor/diagnostic_settings/_create.py", line 176, in _execute_operations
File "D:\a_work\1\s\build_scripts\windows\artifacts\cli\Lib\site-packages\azure/cli/command_modules/monitor/operations/diagnostics_settings.py", line 98, in pre_operations
AttributeError: 'AAZCommandCtx' object has no attribute 'resource'
To Reproduce
create a pipeline to install latest version of azure CLI and try to create a diagnostic settings for SQL database.
run below command
az monitor diagnostic-settings create --name test-sql-diagnostic --resource /subscriptions/342343224-04e1-4e68-45g5-d12064350c42/resourceGroups/db-resource-gp/providers/Microsoft.Sql/servers/test-sql/databases/SQL_DB --logs '[{""category"":""SQLInsights"",""enabled"": true},{""category"":""AutomaticTuning"",""enabled"": true},{""category"":""QueryStoreRuntimeStatistics"",""enabled"": true},{""category"":""Errors"",""enabled"": true},{""category"":""DatabaseWaitStatistics"",""enabled"": true},{""category"":""Timeouts"",""enabled"": true},{""category"":""Blocks"",""enabled"": true},{""category"":""Deadlocks"",""enabled"": true}]' --metrics '[{""category"":""Basic"", ""enabled"":true}, {""category"":""InstanceAndAppAdvanced"", ""enabled"":true}, {""category"":""WorkloadManagement"", ""enabled"":true}]' --workspace /subscriptions/43432-04e1-4e68-2313-d12064350c42/resourcegroups/adl-storage-westeurope-rg/providers/Microsoft.OperationalInsights/workspaces/adl-test-48878-oms
Expected behavior
should create a database diagnostic settings.

@kairu-ms for awareness

Same as https://github.com/Azure/azure-cli/issues/24840


Production Caching Issue

There seems to be an issue with production caching. Wondering if we can possibly blow up the project on netlify and rebuild.
It happens mostly with the intial navbar. When you land on https://www.code4puertorico.org/ you'll notice that the navbar links are out of whack. Then if you click on any of the links, when the DOM refreshes, the navbar is fine. For some reason its only when the page initially loads that an old version of the navbar is preserved somewhere and it seems to be at the app level.
@jpadilla Wondering if there is a cache folder created in production that we can blow up and redeploy to possibly solve the issue.

https://github.com/Code4PuertoRico/cfpr-website/pull/23 fixes the root cause


Cannot create directory[/opt/homebrew-cask] when running chef-client as a non-root user

When ensuring directories, the cask recipe makes the assumption that chef-client has been run either as root or with elevated privileges. If running chef-client in local mode as the current user, I run into the following error.
================================================================================
Error executing action `create` on resource 'directory[/opt/homebrew-cask]'
================================================================================

Chef::Exceptions::InsufficientPermissions
-----------------------------------------
Cannot create directory[/opt/homebrew-cask] at /opt/homebrew-cask due to insufficient permissions

Resource Declaration:
---------------------
# In /Users/glevine/.chef/local-mode-cache/cache/cookbooks/homebrew/recipes/cask.rb

 31: directory '/opt/homebrew-cask' do
 32:   owner node['homebrew']['owner'] || homebrew_owner
 33:   mode 00775
 34: end
 35:

Compiled Resource:
------------------
# Declared in /Users/glevine/.chef/local-mode-cache/cache/cookbooks/homebrew/recipes/cask.rb:31:in `from_file'

directory("/opt/homebrew-cask") do
  action :create
  retries 0
  retry_delay 2
  default_guard_interpreter :default
  path "/opt/homebrew-cask"
  declared_type :directory
  cookbook_name "homebrew"
  recipe_name "cask"
  homebrew_owner 501
  owner 501
  mode 509
end

[2015-04-28T23:06:01-04:00] INFO: Running queued delayed notifications before re-raising exception
[2015-04-28T23:06:01-04:00] DEBUG: Re-raising exception: Chef::Exceptions::InsufficientPermissions - directory[/opt/homebrew-cask] (homebrew::cask line 31) had an error: Chef::Exceptions::InsufficientPermissions: Cannot create directory[/opt/homebrew-cask] at /opt/homebrew-cask due to insufficient permissions

The java cookbook includes the homebrew::cask recipe in its homebrew recipe (https://github.com/agileorbit-cookbooks/java/blob/master/recipes/homebrew.rb). So when installing Java using Homebrew, it seems like it is a requirement to run chef-client as root. This has two smells to me:

The user is effectively required to install Java as root, which runs counter to how Homebrew is designed.
Even if I wrote a wrapper script to create the directory before calling chef-client, then I would still run into this issue anytime I tried to use the java community cookbook.

The java cookbook is just an example, though. This could be true for any dependent cookbook. But it is especially difficult to work around when trying to use community cookbooks.
Maybe this could be passed off as an issue with the java cookbook. But it just feels like cookbooks dependent on the homebrew cookbook are not expecting this cookbook to behave this way -- if being permission agnostic is a goal.

If running chef-client in local mode as the current user, I run into the following error.

Yes. This is actually expected, as /opt is not writable by the normal user. A thing that Homebrew and Homebrew Cask do is shell out with sudo to create the directories they need.

Homebrew install script
Homebrew Cask

When running these recipes with Chef, it's possible that sudo is executed transparently when they're installing, or it might prompt for a password. Either way, while homebrew recommends against doing things with sudo, the fact is you still need to do certain  things on your system with elevated privileges. For example, when you install software from a .pkg, and you get prompted for your password by OS X, that's a sudo prompt that it's going to use behind the GUI to perform the installation.

In my case, I already have Homebrew and Cask installed. So when testing a recipe, I wasn't expecting this failure. If I run chef-client in local mode as myself and /opt/homebrew-cask already exists (and is owned by me), then I would expect everything to run just fine. If that directory didn't already exist in such a state, then I would totally understand the error.
The Cask installer requires sudo to create its directories, but then it modifies the permissions of everything except /opt so that sudo isn't necessary for installations of software. My point about it being counter to Homebrew's recommendations was only that after installation of the tool, sudo should no longer be necessary.
So I guess the real issue is that the cask recipe isn't taking into account whether or not those directories already exist and are in the correct state. The default recipe uses a not_if (https://github.com/opscode-cookbooks/homebrew/blob/master/recipes/default.rb#L37) to prevent a second attempt at installing Homebrew. It may not be as trivial, but I feel like the the cask recipe should avoid repeating installation steps, too.
Maybe the root cause is in the way that directory works and is thus an issue with chef itself? It appears to attempt to make the directory as opposed to testing its state first. Although, that is just speculation on my part.

+1 Same problem here.  /opt/homebrew-cask already exists, is already owned by the homebrew user.  And yes, it's because the chef directory resource provider checks permissions of the parent directory before it checks whether it even needs to take action.

Is it best to open an issue against chef for the directory resource provider? I just don't know enough about the existing use cases of that resource provider to know if it would open a can of worms.

@jtimberman Do you have an opinion about how best to handle this?

I have been having this same problem as well. Do we even need the following code? Doesn't the Cask installer create these directories if they don't exist?
directory '/Library/Caches/Homebrew/Casks' do
  owner homebrew_owner
  mode 00775
  only_if { ::Dir.exist?('/Library/Caches/Homebrew') }
end

directory '/opt/homebrew-cask' do
  owner homebrew_owner
  mode 00775
  recursive true
end

directory '/opt/homebrew-cask/Caskroom' do
  owner homebrew_owner
  mode 00775
end

Relatedly the default Caskroom location has moved per chef-cookbooks/homebrew#100 and caskroom/homebrew-cask#21603

We no longer manage this directory in the latest version so I'm going to close this out


Why does the title of the article describe gen1, but the content of the article seems to be about gen2？

Hi guys,
We have some data stored on ADLS gen1 and I want to programmatically read the data through Java. But when I develop the demo according to the instructions of the article, I encounter various errors. I took a closer look at what's in the documentation and found that this article seems to be about ADLS gen2 (despite the title being about gen1).
I've developed a demo based on Python's documentation before and didn't encounter so many problems.
Does anyone know what's going on here?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b076d835-855f-a952-ae4b-b6e17ed33d12
Version Independent ID: 197569b3-e6e0-ce75-31a5-d1cb7410c8a5
Content: Java SDK - Filesystem operations on Data Lake Storage Gen1 - Azure
Content Source: articles/data-lake-store/data-lake-store-get-started-java-sdk.md
Service: data-lake-store
GitHub Login: @normesta
Microsoft Alias: normesta

Gadzooks! This is rather confusing.  Thank you for bringing this issue to our attention @fsgdgsdfgsgd  .  I agree with your assessment, I see references to both Gen1 and Gen2 in this document.  It has been a while since I last used Gen1, but I recall Gen1 and Gen2 being different enough to not be interchangeable.  If there were just a couple typos I would change it myself, but there is enough here I need to hand off to the documentation team for review.

Thank you for your feedback. This has been assigned to the content owner, @normesta , and will be updated as appropriate.
@normesta  there are enough incongruent contents blended together in this document, I am no longer sure which belong.  This needs attention to detail in repair.  This is also severely confusing to readers.

Agreed. I've reached out to the committer for clarity on the commit.  I'll have to likely revert the last commit. The last commit updates the SDK to use v12, but for Gen1, I don't believe v12 of the SDK applies.

Thanks for your quick reply.
I just checked the commit history of this document. Is this the gen 1 document? Azure Data Lake Store libraries for Java Because I see the same SDK and sample code as the document before modification.
But I noticed that the authentication method of this sample code is very different from the SDK of other languages.
Java:
// AccessTokenProvider provider = new ClientCredsTokenProvider(authTokenEndpoint, clientId, clientKey);
ADLStoreClient client = ADLStoreClient.createClient(accountFQDN, provider);

Python:
tenant = '<TENANT>'
RESOURCE = 'https://datalake.azure.net/'
client_id = '<CLIENT_ID>'
client_secret = '<CLIENT_SECRET>'

adlCreds = lib.auth(tenant_id = tenant,
                client_secret = client_secret,
                client_id = client_id,
                resource = RESOURCE)

C#:
private static void Main(string[] args)
{
    // Service principal / application authentication with client secret / key
    // Use the client ID of an existing AAD "Web App" application.
    string TENANT = "<AAD-directory-domain>";
    string CLIENTID = "<AAD_WEB_APP_CLIENT_ID>";
    System.Uri ARM_TOKEN_AUDIENCE = new System.Uri(@"https://management.core.windows.net/");
    System.Uri ADL_TOKEN_AUDIENCE = new System.Uri(@"https://datalake.azure.net/");
    string secret_key = "<AAD_WEB_APP_SECRET_KEY>";
    var armCreds = GetCreds_SPI_SecretKey(TENANT, ARM_TOKEN_AUDIENCE, CLIENTID, secret_key);
    var adlCreds = GetCreds_SPI_SecretKey(TENANT, ADL_TOKEN_AUDIENCE, CLIENTID, secret_key);
}

The authentication methods for C# and Python appear to be the same. It asks me to provide a tenant id, client id and client secret to generate a cred, then use this cred and my adls account name to create a client.
For Java, what is the authTokenEndpoint? I thought it was a string like https://login.microsoftonline.com/<tenant_id>/oauth2/v2.0/token at first. But I encountered this error:
Error enumerating directory /A/B
Error fetching access token
Operation null failed with exception java.io.IOException : Failed to acquire token from AzureAD. Http response: 400 Bad Request
Last encountered exception thrown after 5 tries. [java.io.IOException,java.io.IOException,java.io.IOException,java.io.IOException,java.io.IOException]
 [ServerRequestId:null]

Another thing is the document about service-to-service authentication seems to have the same problem. Service-to-service authentication with Azure Data Lake Storage Gen2 using Java It is placed under the directory of gen1, but the content is about gen2.

Ok content has been reverted. Looks like we a well-intentioned PR from a contributor somehow made it through. Sorry for the confusion.
#please-close


Reduce the die size by 8 bytes

Description
The heavily used die is currently 96 bytes. With this PR, it is brought down to 88 bytes, by removing the _hash member. We still need the hash to store the die in the global map, but it can be computed on demand.

When the fatal attribute hashing PR lands, this one will be hard to merge. The problem is that deriving the hash requires the attributes of the die, which are no longer stored with the die in the FAH branch. We can still get this hash on an as-needed basis, but it will require going back to disk to re-read the die's attributes. We'll need to re-evaluate the tradeoffs here once the FAH branch lands.

Good point; we'll hold off on this and look again after FAH.

A wash to the FAH work.


Reuse Projected Hamiltonian Tensors

It occurs to me the current setup of using TDVP has a major inefficiency: after calling the tdvp function on a given timestep, for the next timestep the exact same MPS that was returned is often reused as the starting state for the next timestep. This means that many of the "environment" tensors representing the Hamiltonian projected into the MPS basis are being recomputed two times when they could be computed just once.
Some solutions could be:

enable saving the projection tensors and passing them back to the algorithm each time
making a second, higher-level interface where one specifies the total time value and then that code runs TDVP where a certain amount of time evolution is done each nsweeps sweeps

Could you be more explicit with some code (for example pointing out problematic lines of code or showing a minimal example)? Is this an issue within the tdvp function, or when you call the tdvp function multiple times?

For example you should be able to do:
tdvp(H, ψ0, -0.1im; nsweeps=10)

in order to evolve to time 1.0, does that address the problem you are seeing?

Currently the time that is input is the timestep per sweep, which was an arbitrary choice. We can change it so that the input time t is the total time, and the timestep is t/nsweeps.
Agreed it would be nice to split out the code that happens within a sweep into a separate function (say tdvp_iteration or tdvp_sweep). That could accept and return the environment tensors to make it more efficient, as you suggest. That is the design of the TDVP/VUMPS code in ITensorInfiniteMPS.jl.

Thanks for clarifying these things. I was actually thinking it could be a bug that the total time is t*nsweeps in the current code, and that one might have reasons to do more than one sweep per time step.
But perhaps that is a rare, if ever needed feature and we should leave it as it is now where the total time evolved is t*nsweeps.
In that case, then yes mainly what would be needed is an interface change or a second interface where you input a total time and then it figures out the number of sweeps to do.
Finally, one drawback of that sort of thing is that if one wants to "monitor" some property as the time evolution is proceeding, then one has to use an observer or that sort of thing. This can actually get rather complicated: one algorithm I'm currently doing involves time-evolving 4 MPS at once, and overlapping them every time step to get the desired result.
So then one idea is that we can think of ways to make "anonymous observers" – something like this:
psi = tdvp

To make sure I answered your question above, I was only referring to the case of calling tdvp multiple times in a loop.

Right, it sounds like a simple way to go is to have a high level interface tdvp that does multiple sweeps of TDVP up to a desired number of sweeps/total time (like the current version does), with the possibility of passing an observer like we do with DMRG. That would be the high level interface.
Then we could have a version which just calls a single sweep of TDVP (maybe called tdvp_iteration or tdvp_sweep) which is the "low level" interface for more advanced requirements like the one you mentioned of time evolving 4 different states at the same time. This version would output the projected MPOs so they can be passed to the next iteration. Additionally, the high level tdvp function would be implemented just by looping over tdvp_iteration/tdvp_sweep. Does that sound reasonable?

Yes, let's go for that design. Sounds like the right choice.
Seperately, let's think about how to make really simple observers, like that anonymous example above. I know you and Giacomo had worked on a more simple observer design in the context of PastaQ.

We have a package: https://github.com/GTorlai/Observers.jl which I think is pretty simple to use. Basically you just create an Observer from a list of functions you want called at each iteration, and then you retrieve the results with a dictionary-like interface.

The goal was to generalize the observer functionality in ITensor with just a single type, and ultimately the hope was to just use that Observer type in dmrg. So tdvp in this package would be a good testing ground for that.


R_0 values used in the figure need to be updated if we are referencing the PNAS version of Tim's analysis

https://github.com/azwaans/timtam-diamond-princess/blob/802e531a1c401a0770fede2451c80d81d57fb8eb/R/postprocessing-part-4.R#L195C1-L205C4
We need to update the R_0 values used in the report of Tim's analysis to be consistent with the new PNAS publication.
Based on the published version of the article, these values appear to have been updated. See both Figure 1 of the paper and Figure S4 of the Supplementary Information.
@azwaans, did you get these values from Tim directly? I'm not sure how I would have pulled these from the pre-print otherwise.
Also, I need to update the annotation in the figure.

The update-pnas-r0 branch looks good to me, are you happy with it? If so, let's merge and I'll copy the figures into the manuscript repo :)


[bug] tauri.shell.open does not working on differtent linux distribution than building platform with appimage build

Describe the bug
One of user told me that buttons opens brower does not works on Arch Linux with AppImage built on GitHub Actions.
After small investigation, open function become not working if running on other distributions than the AppImage was built.
Reproduction
Downloading https://github.com/anatawa12/tauri-link-appimage-test/releases/tag/app-v0.1.0 and run on Arch Linux will reproduce this problem and downloading https://github.com/vrc-get/vrc-get/issues/934#issuecomment-2164885456 and run on Ubuntu 24.04 will also reproduce this problem
Expected behavior
It should works
Full tauri info output
[✔] Environment
    - OS: Mac OS 14.4.1 X64
    ✔ Xcode Command Line Tools: installed
    ✔ rustc: 1.79.0 (129f3b996 2024-06-10)
    ✔ cargo: 1.79.0 (ffa9cf99a 2024-06-03)
    ✔ rustup: 1.27.1 (54dd3d00f 2024-04-24)
    ✔ Rust toolchain: stable-aarch64-apple-darwin (default)
    - node: 20.12.2
    - pnpm: 8.15.4
    - yarn: 1.22.19
    - npm: 10.8.1

[-] Packages
    - tauri [RUST]: 1.6.8
    - tauri-build [RUST]: 1.5.2
    - wry [RUST]: 0.24.10
    - tao [RUST]: 0.16.9
    - tauri-cli [RUST]: 1.5.14
    - @tauri-apps/api [NPM]: 1.5.6
    - @tauri-apps/cli [NPM]: 1.5.14

[-] App
    - build-type: bundle
    - CSP: unset
    - distDir: out
    - devPath: http://localhost:3000/
    - framework: React (Next.js)
    - bundler: Webpack

Stack trace
No response
Additional context
Downstream issue: vrc-get/vrc-get#934

According to @hypevhs in vrc-get/vrc-get#1251, bundling xdg-open script causes this problem so it might be good to have option to disable bundling xdg-open?
I feel it's better to use xdg-open of user environment than using it of build environment.

I feel it's better to use xdg-open of user environment than using it of build environment.

This is not really how appimages work but i guess with the amount of issues we have with appimages it makes sense to expose this to devs. Should be a pretty simple change, we can just check if the APPIMAGE_BUNDLE_XDG_OPEN env var is already set in the cli before setting it to 1

Can I ask, what were the problems for which the solution was to copy in xdg-open instead of trusting the host to provide it? Given that xdg-open usually just forwards the request to some other executable on the host (kde-open, gio), I can't think of a reason why including it would improve compatibility.

I'm away over the weekend so I can't link any sources (idk if they were public), but the main reason was that we actually got reports from users that did not have xdg-open installed. I doubt that this was their last problem or that bundling xdg-open helped that much, but since that's how appimages work we simply opted for bundling it.
I could also swear that a user had version mismatch issues without this change but that system was probably just messed up or I'm mixing things up 2 yrs later 🤷

I just tried Lettura (RSS reader with an AppImage and a few code paths that call shell.open) and it has the same issue. So it's not that shell.open is broken in some tauri apps, it's that it's broken in all tauri apps for some users. We could certainly still add the build-time opt-out switch, or even provide a similar runtime opt-out switch for users, but rather than wait for the devs & users of each app to stumble over this issue, I interpret this as an opportunity to rework the default behavior.
Shell.open could prefer the host's xdg-open, and if it doesn't exist, fall back to the bundled copy. WDYT?

Hmm yeah, I wouldn't mind that. Do you have an implementation approach in mind? Just hardcode /usr/bin/xdg-open?

Maybe instead of hardcoding a full path, could just name the bundled copy "xdg-open-fallback". Try to run xdg-open, and if not found, run xdg-open-fallback.

After workarounding this issue by using open crate directly, kde-oen invoked in xdg-open become not working due to Library version conflict with bundled one and system one so we also should fix problem I think.
In my app, by removing AppDir entries from LD_LIBRARY_PATH, I workarounded thia problem.
https://github.com/vrc-get/vrc-get/pull/1260#issuecomment-2242153215

+1 on opt-out of xdg-open bundling. It gets exponentially more complicated, because I get different behaviors with:

AppImage built locally on my dev machine (later Ubuntu)
AppImage built in CI which uses old base image as per recommendations (do I need to have xdg-utils installed? - it's not documented).
At runtime, depending on what the user has

I've seen everything from silent failures on some file types, and urls, to errors, to opening in different applications (Chrome instead of Firefox, for me), to different behavior based on changing XDG_DATA_DIRS env var.
It's a mess, and this isn't Tauri's fault. But, I would say that it's much preferable to deal with explicit errors (which later versions of xdg-open give, and also if xdg-open is not available). At least then we can tell the user "hey we tried to open this link, but it didn't work so here you can open it manually".
Sure, there might be a set of users who don't have xdg-utils installed. But the version on the old ubuntu base image (20.04) is so bad and inconsistent that I'd rather have it work for most/modern/correctly set up systems, and provide some troubleshooting docs on the side.

opened a pr earlier today, forgot to link this issue tho https://github.com/tauri-apps/plugins-workspace/pull/2103
for now it's just really simply trying to call /usr/bin/xdg-open on the system and not inside the appimage (falling back to the old behavior)
Another thing that we need to check there is the mentioned XDG_DATA_DIRS env var, this one iirc is also a problem in appimages (because the runtime sets it to something appimage specific).
In theory the more correct approach would be to use an exec wrapper but i never was able to integrate them without causing even more issues (especailly due to webkitgtk) so i hope the simple approach here works as well.
btw i'd appreciate it if someone could test the pr
as long as we're using webkitgtk inside appimages (it really doesn't want to), better solutions are not worth my time at the moment.

@FabianLars awesome, thanks for linking the issues.
Reading up a bit: https://www.freedesktop.org/wiki/Software/xdg-utils/#usingxdg-utils
TL;DR: They actually do recommend bundling the xdg utils.
There could be issues with hardcoding /usr/bin/xdg-open, when it should probably be found in the PATH. However, does anyone understand the AppImage specifics around this? If AppImage relies on a self-contained notion of an app, perhaps the right way is actually bundling.. The issue is the old version, that fails silently and other stuff.
If we can bundle the new version (it is backward compatible), perhaps we should do that?
See also: https://github.com/AppImage/AppImageKit/issues/396
I'm gonna ask the appimage folks what they think.

Unfortunately this was really hard and I couldn't find a solution. But I will share my research:

AppImage runtime sets and/or alters a bunch of env vars, PATH, XDG_DATA_DIRS, LD_LIBRARY_PATH etc. God knows what.
This env is used when starting subprocesses, by default

On my Ubuntu 22.04 local machine with Tauri 1.8.1, xdg-open 1.1.3, this results in:

I didn't have xdg-open bundled in the appimage dir, suggesting it's not bundled by Tauri
Opening files works, suggesting my system xdg-open is found
...but URLs simply don't open. I do not get a JS error.

I've found some more issues/writing about this, and seemingly the suggestion is to restore/clean up the env prior to launching. I have not verified this myself:

One solution is to restore the env through /proc/[pid]/environ on the parent process, before launching the subprocess.
Another one is to remove all known AppImage prefixes from the environment with regex

Personally, I went with a different workaround:

Make the shell.open(url) call
Wait 1s
Check window focus. If still in focus, copy the URL to clipboard and notify the user so they can manually paste it.

This is obviously very dumb. But, if the link can't be opened, it's very serious. And since there's a risk of silent errors here, I'd rather take the false positives and allow users to make progress.


[SPARK-24994][SQL] Simplify casts for literal types

What changes were proposed in this pull request?

Currently, in cases like the following:
SELECT * FROM t WHERE age < 40

where age is of short type, Spark won't be able to simplify this and can only generate filter cast(age, int) < 40. This won't get pushed down to datasources and therefore is not optimized.
This PR proposes a optimizer rule to improve this when the following constraints are satisfied:

input expression is binary comparisons when one side is a cast operation and another is a literal.
both the cast child expression and literal are of integral type (i.e., byte, short, int or long)

When this is true, it tries to do several optimizations to either simplify the expression or move the cast to the literal side, so
result filter for the above case becomes age < cast(40 as smallint). This is better since the cast can be optimized away later and the filter can be pushed down to data sources.
This PR follows a similar effort in Presto (https://prestosql.io/blog/2019/05/21/optimizing-the-casts-away.html). Here we only handles integral types but plan to extend to other types as follow-ups.
Why are the changes needed?

As mentioned in the previous section, when cast is not optimized, it cannot be pushed down to data sources which can lead
to unnecessary IO and therefore longer job time and waste of resources. This helps to improve that.
Does this PR introduce any user-facing change?

No.
How was this patch tested?

Added unit tests for both the optimizer rule and filter pushdown on datasource level for both Orc and Parquet.

Test build #127969 has started for PR 29565 at commit a39b99a.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/32595/
Test PASSed.

cc @dbtsai @holdenk @dongjoon-hyun @cloud-fan

Test build #127969 has finished for PR 29565 at commit a39b99a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/127969/
Test PASSed.

Thanks for the PR, @sunchao. This optimization rule will be very useful since many simple predicates with casting as you mentioned are not currently able to pushdown.

Test build #128001 has started for PR 29565 at commit fc4311b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/32627/
Test PASSed.

Test build #128001 has finished for PR 29565 at commit fc4311b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128001/
Test PASSed.

I wondered how it handles null literal, might want to add a test case like this:
  test("unwrap casts when literal is null") {
    val intLit = Literal.create(null, IntegerType)
    val shortLit = Literal.create(null, ShortType)
    assertEquivalent('a > intLit, 'a > shortLit)
    assertEquivalent('a >= intLit, 'a >= shortLit)
    assertEquivalent('a === intLit, 'a === shortLit)
    assertEquivalent('a <=> intLit, 'a <=> shortLit)
    assertEquivalent('a <= intLit, 'a <= shortLit)
    assertEquivalent('a < intLit, 'a < shortLit)
  }

I wondered how it handles null literal, might want to add a test case like this:
  test("unwrap casts when literal is null") {
    val intLit = Literal.create(null, IntegerType)
    val shortLit = Literal.create(null, ShortType)
    assertEquivalent('a > intLit, 'a > shortLit)
    assertEquivalent('a >= intLit, 'a >= shortLit)
    assertEquivalent('a === intLit, 'a === shortLit)
    assertEquivalent('a <=> intLit, 'a <=> shortLit)
    assertEquivalent('a <= intLit, 'a <= shortLit)
    assertEquivalent('a < intLit, 'a < shortLit)
  }

Sure. I can add tests for this. But notice that nulls should have already been handled by NullPropagation though so the binary comparisons should have been transformed to null or something else.

Test build #128032 has started for PR 29565 at commit c813291.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/32659/
Test PASSed.

Test build #128033 has started for PR 29565 at commit 1a7a3e1.

Test build #128033 has finished for PR 29565 at commit 1a7a3e1.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Test build #128032 has finished for PR 29565 at commit c813291.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128033/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128032/
Test FAILed.

retest this please

Test build #128035 has started for PR 29565 at commit 1a7a3e1.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/32661/
Test PASSed.

Test build #128035 has finished for PR 29565 at commit 1a7a3e1.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128035/
Test PASSed.

Test build #128128 has started for PR 29565 at commit 1921601.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/32752/
Test PASSed.

Test build #128128 has finished for PR 29565 at commit 1921601.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128128/
Test PASSed.

Test build #128157 has started for PR 29565 at commit b5371c5.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/32782/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/32790/
Test PASSed.

Test build #128165 has started for PR 29565 at commit aec3053.

Test build #128157 has finished for PR 29565 at commit b5371c5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128157/
Test PASSed.

Test build #128165 has finished for PR 29565 at commit aec3053.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128165/
Test PASSed.

@cloud-fan @viirya @dongjoon-hyun can you take another look before I merge it? Thanks.

@sunchao there is a conflict. Can you rebase it? Thanks.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33065/
Test PASSed.

Test build #128441 has started for PR 29565 at commit 265c169.

Test build #128441 has finished for PR 29565 at commit 265c169.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128441/
Test PASSed.

Test build #128538 has started for PR 29565 at commit 1f87c37.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33163/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33164/
Test PASSed.

Test build #128539 has started for PR 29565 at commit ec88961.

Test build #128538 has finished for PR 29565 at commit 1f87c37.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128538/
Test FAILed.

Test build #128539 has finished for PR 29565 at commit ec88961.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128539/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33202/
Test PASSed.

Test build #128577 has started for PR 29565 at commit de911da.

Test build #128581 has started for PR 29565 at commit 5f32ee5.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/33206/
Test PASSed.

Test build #128577 has finished for PR 29565 at commit de911da.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128577/
Test PASSed.

Test build #128581 has finished for PR 29565 at commit 5f32ee5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/128581/
Test PASSed.

Thank you all for the review and commit!

Late LGTM with one minor comment: https://github.com/apache/spark/pull/29565/files#r487646973


chore: Remove locales from docs.microsoft.com urls

Why:
Microsoft links automatically direct to the users locale when it is removed from the URLs. This is also done for the docs.microsoft.com content in order to improve localization of content. There were changes to the translated content in the cleanup, but it wasn't committed here, so it would flow through the regular translation flow.
What's being changed:
Ran the cleanup from https://github.com/microsoft/vscode-docs-authoring/ to strip locales from MS urls
Check off the following:

[x] All of the tests are passing.
[x] I have reviewed my changes in staging.
[x] For content changes, I have reviewed the localization checklist
[x] For content changes, I have reviewed the Content style guide for GitHub Docs.

Thanks @nschonni! I'll triage this for a content review, as well.


Write a JavaScript program to print reverse of a number

Description
Write a JavaScript program to print reverse of a number
Reverse of a number is the number obtained by reversing the digits of the original number. For example, the reverse of 123 is 321. The reverse of 100 is 1. The reverse of 1000 is 1.
Input  : 123
Output : 321

How to contribute

Save the solution in program/PrintReverseOfANumber.js file
Add PrintReverseOfANumber.js file in print-reverse-of-a-number folder

!assign


ORA-00922: missing or invalid option occurred when manually sourcing the Oracle sql script

Enviornment
DB - Oracle 12.
wso2is-5.9.0-alpha pack
Priority: High
Severity: High
Steps:
Took the latest oracle script
Source that oracle (/wso2is-5.9.0-alpha/dbscripts)
Error starting at line : 32 in command -
CREATE TABLE REG_PATH(
REG_PATH_ID INTEGER NULL,
REG_PATH_VALUE VARCHAR2(2000) NOT NULL,
REG_PATH_PARENT_ID INTEGER,
REG_TENANT_ID INTEGER DEFAULT 0,
CONSTRAINT PK_PATH PRIMARY KEY(REG_PATH_ID, REG_TENANT_ID)),
CONSTRAINT UNIQUE_REG_PATH_TENANT_ID UNIQUE (REG_PATH_VALUE,REG_TENANT_ID)
Error report -
ORA-00922: missing or invalid option
00922. 00000 -  "missing or invalid option"
*Cause:
*Action:
It should be corrected as follows.
CREATE TABLE REG_PATH(
REG_PATH_ID INTEGER NULL,
REG_PATH_VALUE VARCHAR2(2000) NOT NULL,
REG_PATH_PARENT_ID INTEGER,
REG_TENANT_ID INTEGER DEFAULT 0,
CONSTRAINT PK_PATH PRIMARY KEY(REG_PATH_ID, REG_TENANT_ID),
CONSTRAINT UNIQUE_REG_PATH_TENANT_ID UNIQUE (REG_PATH_VALUE,REG_TENANT_ID))
/
Because of this ,
Error starting at line : 40 in command -
CREATE INDEX REG_PATH_IND_BY_PATH_VALUE ON REG_PATH(REG_PATH_VALUE, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 42 in command -
CREATE INDEX REG_PATH_IND_BY_PARENT_ID ON REG_PATH(REG_PATH_PARENT_ID, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 46 in command -
CREATE OR REPLACE TRIGGER REG_PATH_TRIGGER
BEFORE INSERT
ON REG_PATH
REFERENCING NEW AS NEW
FOR EACH ROW
BEGIN
SELECT REG_PATH_SEQUENCE.nextval INTO :NEW.REG_PATH_ID FROM dual;
END;
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 81 in command -
CREATE TABLE REG_RESOURCE (
REG_PATH_ID         INTEGER NOT NULL,
REG_NAME            VARCHAR2(256),
REG_VERSION         INTEGER,
REG_MEDIA_TYPE      VARCHAR2(500),
REG_CREATOR         VARCHAR2(31) NOT NULL,
REG_CREATED_TIME    TIMESTAMP NOT NULL,
REG_LAST_UPDATOR    VARCHAR2(31),
REG_LAST_UPDATED_TIME    TIMESTAMP NOT NULL,
REG_DESCRIPTION     VARCHAR2(1000),
REG_CONTENT_ID      INTEGER,
REG_TENANT_ID INTEGER DEFAULT 0,
REG_UUID VARCHAR2(100) NOT NULL,
CONSTRAINT FK_REG_RES_PATH FOREIGN KEY (REG_PATH_ID,REG_TENANT_ID) REFERENCES REG_PATH (REG_PATH_ID,REG_TENANT_ID),
CONSTRAINT PK_REG_RESOURCE PRIMARY KEY(REG_VERSION, REG_TENANT_ID))
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Sequence REG_RESOURCE_SEQUENCE created.
Error starting at line : 99 in command -
CREATE OR REPLACE TRIGGER REG_RESOURCE_TRIGGER
BEFORE INSERT
ON REG_RESOURCE
REFERENCING NEW AS NEW
FOR EACH ROW
BEGIN
SELECT REG_RESOURCE_SEQUENCE.nextval INTO :NEW.REG_VERSION FROM dual;
END;
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 109 in command -
CREATE INDEX REG_RESOURCE_IND_BY_NAME ON REG_RESOURCE(REG_NAME, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 111 in command -
CREATE INDEX REG_RESOURCE_IND_BY_PATH_ID ON REG_RESOURCE(REG_PATH_ID, REG_NAME, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 113 in command -
CREATE INDEX REG_RESOURCE_IND_BY_UUID ON REG_RESOURCE(REG_UUID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 115 in command -
CREATE INDEX REG_RESOURCE_IND_BY_TENAN ON REG_RESOURCE(REG_TENANT_ID, REG_UUID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 117 in command -
CREATE INDEX REG_RESOURCE_IND_BY_TYPE ON REG_RESOURCE(REG_TENANT_ID, REG_MEDIA_TYPE)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 119 in command -
CREATE TABLE REG_RESOURCE_HISTORY (
REG_PATH_ID         INTEGER NOT NULL,
REG_NAME            VARCHAR2(256),
REG_VERSION         INTEGER NOT NULL,
REG_MEDIA_TYPE      VARCHAR2(500),
REG_CREATOR         VARCHAR2(31) NOT NULL,
REG_CREATED_TIME    TIMESTAMP NOT NULL,
REG_LAST_UPDATOR    VARCHAR2(31),
REG_LAST_UPDATED_TIME    TIMESTAMP NOT NULL,
REG_DESCRIPTION     VARCHAR2(1000),
REG_CONTENT_ID      INTEGER,
REG_DELETED         SMALLINT,
REG_TENANT_ID INTEGER DEFAULT 0,
REG_UUID VARCHAR2(100) NOT NULL,
FOREIGN KEY (REG_PATH_ID,REG_TENANT_ID) REFERENCES REG_PATH (REG_PATH_ID,REG_TENANT_ID),
FOREIGN KEY (REG_CONTENT_ID,REG_TENANT_ID) REFERENCES REG_CONTENT_HISTORY (REG_CONTENT_ID,REG_TENANT_ID),
CONSTRAINT PK_REG_RESOURCE_HISTORY PRIMARY KEY(REG_VERSION, REG_TENANT_ID))
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 137 in command -
CREATE INDEX REG_RES_HIST_IND_BY_NAME ON REG_RESOURCE_HISTORY(REG_NAME, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 139 in command -
CREATE INDEX REG_RES_HIST_IND_BY_PATH_ID ON REG_RESOURCE_HISTORY(REG_PATH_ID, REG_NAME, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 161 in command -
CREATE TABLE REG_RESOURCE_COMMENT (
REG_COMMENT_ID          INTEGER NOT NULL,
REG_VERSION             INTEGER,
REG_PATH_ID             INTEGER,
REG_RESOURCE_NAME       VARCHAR2(256),
FOREIGN KEY (REG_PATH_ID,REG_TENANT_ID) REFERENCES REG_PATH (REG_PATH_ID,REG_TENANT_ID),
FOREIGN KEY (REG_COMMENT_ID,REG_TENANT_ID) REFERENCES REG_COMMENT (REG_ID,REG_TENANT_ID),
REG_TENANT_ID INTEGER DEFAULT 0)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 170 in command -
CREATE INDEX REG_RES_COMM_BY_PATH_ID ON REG_RESOURCE_COMMENT(REG_PATH_ID, REG_RESOURCE_NAME, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 172 in command -
CREATE INDEX REG_RES_COMM_BY_VERSION ON REG_RESOURCE_COMMENT(REG_VERSION, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 194 in command -
CREATE TABLE REG_RESOURCE_RATING (
REG_RATING_ID           INTEGER NOT NULL,
REG_VERSION             INTEGER,
REG_PATH_ID             INTEGER,
REG_RESOURCE_NAME       VARCHAR2(256),
FOREIGN KEY (REG_PATH_ID,REG_TENANT_ID) REFERENCES REG_PATH (REG_PATH_ID,REG_TENANT_ID),
FOREIGN KEY (REG_RATING_ID,REG_TENANT_ID) REFERENCES REG_RATING (REG_ID,REG_TENANT_ID),
REG_TENANT_ID INTEGER DEFAULT 0)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 203 in command -
CREATE INDEX REG_RATING_IND_BY_PATH_ID ON REG_RESOURCE_RATING(REG_PATH_ID, REG_RESOURCE_NAME, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 205 in command -
CREATE INDEX REG_RATING_IND_BY_VERSION ON REG_RESOURCE_RATING(REG_VERSION, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 228 in command -
CREATE TABLE REG_RESOURCE_TAG (
REG_TAG_ID              INTEGER NOT NULL,
REG_VERSION             INTEGER,
REG_PATH_ID             INTEGER,
REG_RESOURCE_NAME       VARCHAR2(256),
FOREIGN KEY (REG_PATH_ID,REG_TENANT_ID) REFERENCES REG_PATH (REG_PATH_ID,REG_TENANT_ID),
FOREIGN KEY (REG_TAG_ID,REG_TENANT_ID) REFERENCES REG_TAG (REG_ID,REG_TENANT_ID),
REG_TENANT_ID INTEGER DEFAULT 0)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 237 in command -
CREATE INDEX REG_TAG_IND_BY_PATH_ID ON REG_RESOURCE_TAG(REG_PATH_ID, REG_RESOURCE_NAME, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 239 in command -
CREATE INDEX REG_TAG_IND_BY_VERSION ON REG_RESOURCE_TAG(REG_VERSION, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 228 in command -
CREATE TABLE REG_RESOURCE_TAG (
REG_TAG_ID              INTEGER NOT NULL,
REG_VERSION             INTEGER,
REG_PATH_ID             INTEGER,
REG_RESOURCE_NAME       VARCHAR2(256),
FOREIGN KEY (REG_PATH_ID,REG_TENANT_ID) REFERENCES REG_PATH (REG_PATH_ID,REG_TENANT_ID),
FOREIGN KEY (REG_TAG_ID,REG_TENANT_ID) REFERENCES REG_TAG (REG_ID,REG_TENANT_ID),
REG_TENANT_ID INTEGER DEFAULT 0)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 237 in command -
CREATE INDEX REG_TAG_IND_BY_PATH_ID ON REG_RESOURCE_TAG(REG_PATH_ID, REG_RESOURCE_NAME, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:
Error starting at line : 239 in command -
CREATE INDEX REG_TAG_IND_BY_VERSION ON REG_RESOURCE_TAG(REG_VERSION, REG_TENANT_ID)
Error report -
ORA-00942: table or view does not exist
00942. 00000 -  "table or view does not exist"
*Cause:
*Action:

PR Merged,


set Host header using setExtraHTTPHeaders net::ERR_INVALID_ARGUMENT

Steps to reproduce

Puppeteer version: 5.0.0

What steps will reproduce the problem?
const puppeteer = require('puppeteer');

(async () => {
  const browser = await puppeteer.launch({
    args: [
        '--no-sandbox',
        '--headless',
        '--disable-gpu',
        '--user-agent=Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:78.0) Gecko/20100101 Firefox/78.0'
    ]
  });
  await new Promise(resolve => setTimeout(resolve, 5000));
  const page = await browser.newPage();
  await page.setExtraHTTPHeaders({'Host': 'bukdjango_captcha.com'})
  await page.goto('http://bukdjango_captcha.com:8000/recaptcha_v3');
  await page.screenshot({path: 'example.png'});

  console.log(await page.content())
    await browser.close();
})();

What is the expected result?
I can override host header.
What happens instead?
tests_browser_1      | (node:1) UnhandledPromiseRejectionWarning: Error: net::ERR_INVALID_ARGUMENT at http://bukdjango_captcha.com:8000/recaptcha_v3
tests_browser_1      |     at navigate (/dockertestapp/node_modules/puppeteer/lib/cjs/common/FrameManager.js:96:23)
tests_browser_1      |     at processTicksAndRejections (internal/process/task_queues.js:93:5)
tests_browser_1      |     at async FrameManager.navigateFrame (/dockertestapp/node_modules/puppeteer/lib/cjs/common/FrameManager.js:71:21)
tests_browser_1      |     at async Frame.goto (/dockertestapp/node_modules/puppeteer/lib/cjs/common/FrameManager.js:296:16)
tests_browser_1      |     at async Page.goto (/dockertestapp/node_modules/puppeteer/lib/cjs/common/Page.js:749:16)
tests_browser_1      |     at async /dockertestapp/test.js:16:3
tests_browser_1      |   -- ASYNC --
tests_browser_1      |     at Frame.<anonymous> (/dockertestapp/node_modules/puppeteer/lib/cjs/common/helper.js:109:19)
tests_browser_1      |     at Page.goto (/dockertestapp/node_modules/puppeteer/lib/cjs/common/Page.js:749:53)
tests_browser_1      |     at Page.<anonymous> (/dockertestapp/node_modules/puppeteer/lib/cjs/common/helper.js:110:27)
tests_browser_1      |     at /dockertestapp/test.js:16:14
tests_browser_1      |     at processTicksAndRejections (internal/process/task_queues.js:93:5)
tests_browser_1      | (Use `node --trace-warnings ...` to show where the warning was created)
tests_browser_1      | (node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise whi
ch was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (r
ejection id: 1)
tests_browser_1      | (node:1) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exi
t code.

got the same issue... you solved it?

@footpatrolqueue no but check if your host have unallowed characters like "_"

        'host':'www.slamjam.com',

@footpatrolqueue try without www

already tried.. didnt work

Got same problem with chromedp on Golang.Did you find any sollution yet?

I am facing same problem any update.

Any update on it? I've got the same issue

bump

same issue here

yea same issue

Try sure including the port number?

       const options = {              args,             headless: false,             ignoreHTTPSErrors: true,             Headers:{               "host":"www.host.com",             }         };
Hey guys! Its me again, this fixed it for me back then. Hope this works for you guys as well:)

Sadly that didnt helped in my case

#5077

@pxCaptcha Thanks man, works! For everyone working with DDEV: you have to set the hostname to your project.tld and then take the internal ddev web-container name (usually ddev-[yourprojectname]-web) as goto url:
`
const browser = await puppeteer.launch({
headless: true,
ignoreHTTPSErrors: true,
Headers:{
"host":"yourproject.localhost"
},
args: [
"--no-sandbox",
"--disable-dev-shm-usage",
"--disable-setuid-sandbox",
"--ignore-certificate-errors",
"--allow-insecure-localhost",
"--disable-gpu",
'--disable-web-security',
]
});
const page = await browser.newPage();
await page.setViewport({width: 1920, height: 1024});
await page.setUserAgent('Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/89.0.4389.90 Safari/537.36');
await page.evaluateHandle('document.fonts.ready');
await page.goto("https://ddev-yourproject-web/de_de/", {
waitUntil: 'networkidle0'
});
await page.screenshot({path: "test.png", fullPage: true});
await browser.close();
`

I see setting Host header is actually disabled by default in chromium source code: see here: https://github.com/chromium/chromium/blob/8abfec7efff916f7056a14632cc66c42ff7c24cd/services/network/public/cpp/header_util.cc#L39 comment says: "Disallow setting the Host header because it can conflict with specified URL and logic related to isolating sites."


Added menu options without network for raspi-tie demos

New Menu options:

Thank you, Jasmin for the updates to the raspberry tie demo, to make it usable in offline mode by default.


Issue a warning if docker build takes longer than 30 secs.

Fixes https://github.com/OpenLiberty/ci.maven/issues/903

Sample output:
[INFO] Building Docker image...
[INFO] docker build -f /var/folders/p1/v7svjg7s6xngg377xtsjxcmc0000gn/T/tempDockerfile6946523769638832910 -t liberty-dev-mode /Users/user1/devExperience/demo-devmode
[WARNING] The docker build command is slower than expected. You may increase performance by adding unneeded files and directories such as any Liberty runtime directories to a .dockerignore file in /Users/user1/devExperience/demo-devmode.


Suggestion - List and Grid View Options can be implemented in the wish-list.

Issue Description
When checking the details of the wishlist products and there we can see that the products are visible in the list view. An option can be implemented similar to Category products and search products, where we can see the wishlist product in the list and grid view.
Bagisto Version - V1.4.1
Steps To Reproduce
1. Add products to the wishlist.
2. Check the wishlist and we can see the products in the list view.
Actual Result: There is no be an option to view the products in the Grid/List view.
Expected Result: There should be an option to view the products in the Grid/List view. Please take reference from Here.

Closing The Issue
Reason - In the Category Section or  Search Section we do have various Products and there it is required but in the case of wishlist we do have limited products, so not required.