Add debug stack trace in wellcome/ml.init

Description
Fixes #261
Now when you set export LOGGING_LEVEL=DEBUG you can get something like
2021-03-25 12:00:07 tensorflow DEBUG: Falling back to TensorFlow client; we recommended you install the Cloud TPU client directly with pip install cloud-tpu-client.
2021-03-25 12:00:12 wellcomeml.logger ERROR: No module named 'torch'
2021-03-25 12:00:12 wellcomeml.logger WARNING: Using WellcomeML without extras (transformers & torch).
2021-03-25 12:00:12 wellcomeml.logger DEBUG: Traceback (most recent call last):
  File "/Users/nsorros/code/WellcomeML/wellcomeml/ml/__init__.py", line 12, in <module>
    from .vectorizer import Vectorizer
  File "/Users/nsorros/code/WellcomeML/wellcomeml/ml/vectorizer.py", line 11, in <module>
  1 Description
    from wellcomeml.ml.bert_vectorizer import BertVectorizer
  File "/Users/nsorros/code/WellcomeML/wellcomeml/ml/bert_vectorizer.py", line 9, in <module>
    import torch
ModuleNotFoundError: No module named 'torch'

Checklist

[x] Added link to Github issue or Trello card
[ ] Added tests

@nsorros This is v handy, but could do with updating the README too so people know about it?

@nsorros This is v handy, but could do with updating the README too so people know about it?

eca794d
How about this? Should we say more or something different? For now, it is only one place where we make use of DEBUG but in the future we might add more of those.

Codecov Report

Merging #262 (eca794d) into main (5f31fac) will decrease coverage by 0.04%.
The diff coverage is 50.00%.

@@            Coverage Diff             @@
##             main     #262      +/-   ##
==========================================
- Coverage   85.65%   85.60%   -0.05%
==========================================
  Files          41       41
  Lines        2237     2237
==========================================
- Hits         1916     1915       -1
- Misses        321      322       +1

Impacted Files
Coverage Δ

wellcomeml/ml/__init__.py
85.18% <50.00%> (-2.82%)
:arrow_down:

wellcomeml/io/s3_policy_data.py
91.17% <0.00%> (-0.13%)
:arrow_down:

wellcomeml/ml/bilstm.py
91.42% <0.00%> (-0.07%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5f31fac...eca794d. Read the comment docs.


Describe plugin loading order

From<EMAIL_ADDRESS>on 2011-11-12T06:22:42Z
What steps will reproduce the problem?
It is not clear how to approach Spyder customization without clear docs on plugin loading process. What is the expected output? What do you see instead? I expect a picture of Spyder internals and loading process to see where I could 'plug in'.
Original issue: http://code.google.com/p/spyderlib/issues/detail?id=833

Spyder plugin order can be seen in spyderlib/spyder.py. Other than that, I don't think we're going to do something else about it.


Custom loader fails to resolve files

Hi all,
I'm confused why my loader is failing to resolve CSS import paths. At least, failing to resolve them consistently with other loaders in the pipeline.
I have only altered slightly this webpack config from react-starter-kit. The module loaders in the /\.scss$/ section have my loader earliest, viz:
//module.loaders: [...
{
  test: /\.scss$/,
  loaders: [
    'isomorphic-style-loader',
    'css-loader?' + (DEBUG ? 'sourceMap&' : 'minimize&') +
    'modules&localIdentName=[name]_[local]_[hash:base64:3]',
    'postcss-loader',
    'strip-json-comments-loader',
  ],
}, //...

Without strip-json-comments-loader, the module imports files correctly relative to the importing document. With it, they are relative to the shell root. After npm run build:
    ERROR in ./~/css-loader?sourceMap&modules&localIdentName=[name]_[local]_[hash:base64:3]!./~/postcss-loader!./~/strip-json-comments-loader!./src/components/App/App.scss
    Module build failed: Error: C:\path\to\project\node_modules\strip-json-comments-loader\index.js!C:\path\to\project\src\components\App\App.scss:3:1: Failed to find '../../../node_modules/normalize.css/normalize.css' from C:\path\to\project
        in [
            C:\path\to\project\node_modules\strip-json-comments-loader\index.js!C:\path\to\project\src\components\App
        ]
        at C:\path\to\project\node_modules\strip-json-comments-loader\index.js!C:\path\to\project\src\components\App\App.scss:3:1

I thought loaders did not have to "know" about the source path, as they're little more than a function transforming an input stream.

This is fixed by adding the ?parser=postcss-scss option to postcss-loader: https://github.com/jonathantneal/precss/issues/49#issuecomment-169146317

Thanks. I'll continue this at the postcss-scss or PreCSS projects; continued here.


Make tap on workspace close trash/toolbox

First pass on #112. This makes it so a tap on the workspace will close the
toolbox and the trash if either is open. This also updates the toolbox's
measure to wrap the contents of the toolbox.
To do:
-Make opening one close the other.
-Make tapping on the background also close.

This change is

+@anmatanm +@rachel-fenichel

Review status: 0 of 5 files reviewed at latest revision, all discussions resolved.

Comments from Reviewable

Do your two todos have issue numbers?

Review status: 0 of 5 files reviewed at latest revision, 1 unresolved discussion.

blocklylib-core/src/main/java/com/google/blockly/android/control/BlocklyController.java, line 501 [r1] (raw file):

     */
    public void initWorkspaceView(final WorkspaceView wv) {
        if (mVirtualWorkspaceView != null) {

Comment: // clear references to old view.
(I think that's what you're doing)

Comments from Reviewable

Review status: 0 of 5 files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable

Review status: 0 of 5 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.

blocklylib-core/src/main/java/com/google/blockly/android/control/BlocklyController.java, line 501 [r1] (raw file):
Previously, rachel-fenichel wrote…
> Comment: // clear references to old view.
>
> (I think that's what you're doing)
>

The reference to mVirtualWorkspaceView and mWorkspaceView get replaced right after this. Are there other references I'm missing?

Comments from Reviewable

Review status: 0 of 5 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.

blocklylib-core/src/main/java/com/google/blockly/android/control/BlocklyController.java, line 501 [r1] (raw file):
Previously, RoboErikG wrote…
> The reference to mVirtualWorkspaceView and mWorkspaceView get replaced right after this. Are there other references I'm missing?

I'm confused about why you are setting the onClickListener to null immediately before replacing the virtual workspace view entirely.

Comments from Reviewable

Review status: 0 of 5 files reviewed at latest revision, 1 unresolved discussion, all commit checks successful.

blocklylib-core/src/main/java/com/google/blockly/android/control/BlocklyController.java, line 501 [r1] (raw file):
Previously, rachel-fenichel wrote…
> I'm confused about why you are setting the onClickListener to null immediately before replacing the virtual workspace view entirely.

Oh, you want me to add a comment. Done. =)

Comments from Reviewable

#112 covers the TODOs.

Review status: 0 of 5 files reviewed at latest revision, 1 unresolved discussion.

Comments from Reviewable


Fix ghost communication

Description
Resolves #

Prevents an issue where cells were sent even when there were no new points to send.
This fix saves the failing mili dataset. We will need to see how it affects the nightly tests.

Type of change

[x] Bug fix~~
[ ] New feature
[ ] Documentation update
[ ] Other

How Has This Been Tested?

This solves the issue for the large mili database on the SCF. It remains to be seen how it will affect the test suite.
I ran the mili tests in parallel and they passed.
Reminders:

Please follow the style guidelines of this project.
Please perform a self-review of your code before submitting a PR and asking others to review it.
Please assign reviewers (see VisIt's PR procedures for more information).

Checklist:

[x] I have commented my code where applicable.~~
[x] I have updated the release notes.~~
[ ] I have made corresponding changes to the documentation.
[x] I have added debugging support to my changes.~~
[ ] I have added tests that prove my fix is effective or that my feature works.
[ ] I have confirmed new and existing unit tests pass locally with my changes.
[ ] I have added new baselines for any new tests to the repo.
[x] I have NOT made any changes to protocol or public interfaces in an RC branch.~~

Looks good, feel free to merge.

merge w/ develop in #20126


chore(main): release 6.10.0

:robot: I have created a release beep boop
6.10.0 (2024-12-15)
Features

add fzf-lua integration for grepping in files and dirs (56b80f3)

This PR was generated with Release Please. See documentation.

:robot: Created releases:

v6.10.0
:sunflower:


Noise in silent parts after voice conversion

▼Issue
Generation of noise in silent parts after voice conversion.
▼Work flow

I created a learning dataset of 300 files by removing silence and separating them using Audacity based on high-quality voice sound sources.
I converted it from high-quality reading sound source.
Intense noise occurs in the silent part.

The training settings were batch size 24, and I tried epoch 50, 200, 500, etc., but it seems to occur consistently.
▼Sample
voice:JPN (children book recitation )

https://uploader.cc/s/hvq38p1etkse2mg3as1t0k12yrxhlpsfc73m6995h80l58qo8nlkqc0t0cnsuokm.mp3

▼Environment

RVC-beta.7z(20230428updated)
Windows10 22H2

Same here, but one strange thing, I trained two models with two datasets. One model has this silent noise issue while the other one does not.

One model has this silent noise issue while the other one does not.
Me too. I have a model that uses a different data set and is less noisy.

The dataset I used to train the model that I posted as a sample had very high quality voices, but there was a little background music, so I removed the background music with Demucs-GUI and split the audio. However, the cause seems to be something else, as I have also generated models that are noisy even on fully voice-only datasets.
Also, I tried post-processing with noise gate, etc., but I couldn't cut it because the dB of the voice and noise were about the same.

This issue (Noise in silent part) seems to have been resolved with the latest update. great job! thank you.


(client) after downloading all the songs the server doesn't load

Whenever connecting to a server, I first am met with downloading the required songs. After that's complete I just see a spinner that keeps on going perpetually. I decided to make sure it was a bug, first I tried running a server on a local machine then connecting to it and seems to work fine. Then I ran a server on a Linux VPS (ubuntu 18.04) and I seem to be running into the same issue. This issue seems to be affecting so far patches <IP_ADDRESS> and <IP_ADDRESS>. When I ran my client on 0.4.4 everything seemed to work fine. To make sure this was a bug I checked with my friend who has an oculus rift, and he was running into the same issues.
Beatsaber Version:
0.10.2p1
Hardware:
HTC Vive
Game edition:
Steam

I did some testing on patch 0.4.5 and it seems not to have the problem. I'm planning on testing with a friend if the issue comes up while a song is playing. This will take me a few hours before I can verify that.


Styles priority in mapfish3

Hi.
I'm trying to print vector layer with different styles for some features. My request is similar this:
spec:
{
	. . .
    "layers": [
        {
            "geoJson": "http://localhost:8080/geoserver/wfs?service=wfs&version=2.0.0&outputFormat=application/json&request=GetFeature&typeNames=ns:typename",
			"type": "geojson",
            "opacity": 1,
            "style": {
                "[IN ('typename.1')]":
				{
					"rotation": "0",
                    "symbolizers": [
                        {
                            "strokeWidth": 3.25,
                            "fillColor": "#000000",
                            "fillOpacity": 0,
                            "type": "polygon",
                            "strokeColor": "#3399cc",
                            "strokeOpacity": 1
                        }
                    ]
                },
                "*": {
					"rotation": "0",
                    "symbolizers": [
                        {
                            "strokeWidth": 3.25,
                            "fillColor": "#000000",
                            "fillOpacity": 0.5,
                            "type": "polygon",
                            "strokeColor": "#3399cc",
                            "strokeOpacity": 0.5
                        }
                    ]
                }
            }
        }
    ]
}

http://pastebin.com/77NhzsuR
I'm expect that style [IN ('typename.1')] applying to one feature and style "*" for others. As a result feature "typename.1" will be transparent filled, and others feature will be filled black color with 0.5 opacity. But in fact for feature 'typename.1' applies all styles ("*" and [IN ('typename.1')]).
Can I get a override a less specific style by more specific or set applying priority for styles?
P.S. I'm sorry for my english. :)

Could you try [NOT IN ('typename.1')] instead of * for the default style?

Could you try [NOT IN ('typename.1')] instead of * for the default style?
It's works fine. I'm used NOT key for exclusion rule as workaround in my case. But I think if amount of specific style rules will be much more then in my example, exclusion rule key will be huge. Maybe it makes sense to create ability to specify default style rule which applies only for features without specific style rules?


Koa Isomorphic App 404's after npm run build

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior, please provide code snippets or a repository:

Copy the Koa custom server app
Create a route that returns some JSON object
Use fetch to retrieve the endpoint and show the JSON in your app
npm run dev
Verify the JSON shows on your screen as expected
npm run build
npm run start and try to load that same piece of data again

Expected behavior
The behavior from npm run dev and npm run start are identical.
System information

OS: macOS and Ubuntu
Version of Next.js: Latest as of this post

Additional context
Seems very odd, I've tried reworking the routes on my server with no luck. Currently blocking my production deploy. 99% sure it's something to do with the build process but I'm not familiar with the NextJS build process (or any build processes, not my jam) so anyone with intricate knowledge would be very helpful.

@NewETown I've tried to follow your steps but was unable to reproduce the issue, it works for me. Can you provide a code example?
Here is what I've tried

cloned the koa-example from this repo

yarn add isomorphic-fetch

modified the following files

server.js
const Koa = require('koa');
const next = require('next');
const Router = require('koa-router');

const port = parseInt(process.env.PORT, 10) || 3000;
const dev = process.env.NODE_ENV !== 'production';
const app = next({ dev });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  const server = new Koa();
  const router = new Router();

  router.get('/a', async (ctx) => {
    await app.render(ctx.req, ctx.res, '/b', ctx.query);
    ctx.respond = false;
  });

  router.get('/b', async (ctx) => {
    await app.render(ctx.req, ctx.res, '/a', ctx.query);
    ctx.respond = false;
  });

  router.get('/test', async (ctx) => {
    ctx.body = { some: 'json' };
  });

  router.get('*', async (ctx) => {
    await handle(ctx.req, ctx.res);
    ctx.respond = false;
  });

  server.use(async (ctx, next) => {
    ctx.res.statusCode = 200;
    await next();
  });

  server.use(router.routes());
  server.listen(port, () => {
    console.log(`> Ready on http://localhost:${port}`);
  });
});

a.js
import React, { Component } from 'react';
import fetch from 'isomorphic-fetch';

export default class ComponentA extends Component {
  state = {};

  setStateAsync(state) {
    return new Promise((resolve) => {
      this.setState(state, resolve);
    });
  }

  async componentWillMount() {
    const response = await fetch('http://localhost:3000/test');
    const data = await response.json();
    await this.setStateAsync({ data });
  }

  render() {
    return <div>{JSON.stringify(this.state)}</div>;
  }
}

@gastonfartek Thanks you! I could not reproduce this either.


chore: Update goblin to 0.22

Coverage remained the same at 0.0% when pulling 4379e3754a38ea7bb01b517f4dba55b57c4ecae2 on ignatenkobrain:patch-5 into f2acd9e9dfbceddf2134f9d78496f95f990f770a on gimli-rs:master.


Remove links to https://github.com/bitcoinops/scaling-book/blob/master/x.payment_batching/payment_batching.md

See https://github.com/bitcoinops/scaling-book/pull/30

Untested ACK


docs: initial Jupyter support for website docs, move config notebook

Adds initial support for rendering notebooks using Quarto. Quarto is a tool for rendering technical documentation. It has specific support for rendering Jupyter notebooks that target the Docsaurus documentation system so it is particularly helpful for us.
Additionally, in this PR I moved the oai_openai_utils notebooks to the main docs section as it is a great notebook for explaining how to setup the LLM config list. I made a small edit to add a TL;DR section at the top of the notebook to get started quickly.
Sidebar with new page:

Example of rendered notebook:

Related issue number
#1339

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (6cf5bb0) 33.19% compared to head (8f6788b) 33.19%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1448   +/-   ##
=======================================
  Coverage   33.19%   33.19%
=======================================
  Files          41       41
  Lines        4950     4950
  Branches     1135     1135
=======================================
  Hits         1643     1643
  Misses       3177     3177
  Partials      130      130

Flag
Coverage Δ

unittests
33.15% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Agreed - @zwd-ms has taken on that task and will be done separately to this PR.

Agreed - @zwd-ms has taken on that task and will be done separately to this PR to ensure it is a holistic solution.

Thanks @zwd-ms!

Shall we move it under "Installation"? I think order wise, it's after installation and before use cases.

Shall we move it under "Installation"? I think order wise, it's after installation and before use cases.

I agree, moved it.


Model retrieved from session and saving to database inside a transaction sometimes not being saved to database.

Laravel Version: 8.83.23
PHP Version: 8.0.6
Database Driver & Version: Maria DB 10.4.19

Description:
Adding an unsaved model to session and then retrieving the same model from session and then saving the model inside a database transaction sometimes results in the model not being saved in the database. This is observed to occur whenever an exception is thrown in the code where the $model->save() is being called.
Steps To Reproduce:
use App\Models\MyModel;
use DB;

// instantiate the model but do not save the model to database
// add the model to session
public function preview()
{
    $model = new MyModel;
    $model->name = 'test';
    session()->put('model',$model);
}

// retrieve the model from the session and save to database
public function submit()
{
    $model = session('model');

    DB::beginTransaction();

    $model->save();

    // sometimes an exception my occur in my code and when the user reloads the page and submit() is called again the model is not being saved in the database
   // assume that the exception is not thrown in the second execution of this submit() function when the user reloads the page
    throw new Exception();

    DB::commit();
}

@nk-square Which session driver are you using?

I'm using the default file session driver.

If the exception is getting throwed, then the transaction is not getting commited. So, what's wrong here?

Say the user submits a form and the submit() function is executed and an exception is thrown showing 500 screen on the browser, the model is not saved to the database which is the expected behavior. However, say the user reloads the page and the page and the submit() function is executed again and this time the exception is not thrown (note that the exception thrown is situational and sometimes it is not thrown), on second execution the submit() function completes successfully but the model is not being saved to database.

Hey there,
Unfortunately we don't support this version anymore. Please check out our support policy on which versions we are currently supporting. Can you please try to upgrade to the latest version and see if your problem persists? If so, please open up a new issue and we'll help you out.
Thanks!


Only consider errors from the library functions in use

Our current checks will look at an entire library and consider it either upgrade safe or not. Due to this lack of granularity, a new function we wanted to introduce in the Address utility library in OpenZeppelin Contracts 3.3 caused the entire library to be unusable with the upgrades plugins. (https://github.com/OpenZeppelin/openzeppelin-upgrades/issues/230)
We should only consider errors from those functions that are in use.
This opens an entire can of worms because we obviously need to consider errors from functions used indirectly (by the functions that are used directly, and so on transitively). The logic to implement this probably shares a lot with #159 so I would consider fixing both of these issues at the same time.

Agree, it is a bit annoying to have a mix of upgrade safe and non-upgrade safe contracts in a project. is there any way for it to just search the contract being deployed? Instead of the entire library?

@kevupton Sorry this is not trivial to fix which is why we haven't done it yet.

Hi, guys. Is this still an issue?
I have my upgradeable contract operating with ERC20 contracts using the SafeMath library which includes the Address utility library. And it also causes my Contract to be considered as not safely deployed by your hardhat-upgrades tool. This is very weird because I don't try to make third-party ERC20 contracts upgradeable.
Should I use SafeMathUpgradeable for interactions with non-upgradeable contracts ?????

I'm not sure I understood the full picture but I think the answer to your question is yes. Can you share an example code snippet?

@frangio I am encountering the same error.
I have UUPS upgradeable contracts imported as dev dependencies, as well as one Transparent proxy upgradeable contract also imported as a dependency.
When I deploy them in my tests for my contract to interact with them, I get an error saying that the UUPS upgradeable contracts are
not upgrade safe.
@openzeppelin/contracts/utils/Address.sol:191: Use of delegatecall is not allowed.

The upgradeable contracts are using AddressUpgradeable... so not sure why this issue is occurring.

I'm not sure I understood the full picture but I think the answer to your question is yes. Can you share an example code snippet?

I have solved it by ignoring the error and silencing the warning.
The issue is, that my Upgradeable contract calls non-upgradeable ERC20 Contracts, and I have the line:
  using SafeERC20 for IERC20;
And because your upgrade-validation tool is not "smart" in the dependencies checking, it throws an error on any use of the library that has Upgradeable analog, even if it is not used as upgradeable

I'm not sure I understood the full picture but I think the answer to your question is yes. Can you share an example code snippet?

I have solved it by ignoring the error and silencing the warning. The issue is, that my Upgradeable contract calls non-upgradeable ERC20 Contracts, and I have the line:  using SafeERC20 for IERC20; And because your upgrade-validation tool is not "smart" in the dependencies checking, it throws an error on any use of the library that has Upgradeable analog, even if it is not used as upgradeable

I can silence the error in my test, but these upgradeable contracts I am interacting are on Mainnet.
So when i deploy my contract, it won't be possible to unsafeAllow.
I'm assuming this will be an issue...

Please post on OpenZeppelin Forum with the affected Solidity code.

Please post on OpenZeppelin Forum with the affected Solidity code.

@frangio Please see here:
https://forum.openzeppelin.com/t/interacting-with-uups-upgradeable-contracts-throwing-contract-is-not-upgrade-safe-use-of-delegatecall-is-not-allowed/32743

Hi ! iam having the same issue when deploy a very basic contract. Is there any workaround ?


data-driven Multi-Fidelity Kriging

Hi, thanks for this nice smt!
I have a few high Fidelity data, and also many low Fidelity data, then how to build a  data-driven Multi-Fidelity Kriging with accurracy validation on high Fidelity data?
How to solve the problem? Could you give some advice?
Is there any example concerning the data-driven Multi-Fidelity Kriging?

Thank you!
Best!

Hi,
Concerning multi fidelity data, you have some examples in the documentation:
https://smt.readthedocs.io/en/stable/_src_docs/applications/mfk.html
and in the notebooks
https://github.com/SMTorg/smt/blob/master/tutorial/SMT_MFK_tutorial.ipynb
https://github.com/SMTorg/smt/blob/master/tutorial/SMT_MFK_Noise.ipynb
Best
Nathalie


Update Readme.md snippets to be populated by tool, added SampleSnippe…

Adds SampleSnippets.cs to Text translator, updates Readme.md and runs the snippets tool to populate code areas from SampleSnippets.cs.

API change check
API changes are not detected in this pull request.


Move yeelight into component

Breaking Changes
Move yeelight from light platform to component. As more yeelights are being released, with more features. It will make possible to support them. For example adding sensors for ceiling light, to get current power mode ( daylight / nightlight ). Or switches to support turning on off moonlight ( without having to use service call ). Also switch to turn off both main light and ambilight ( ceiling 650 ). Probably something more, as we will see what ideas we can make for better UX and support.
If you have following configuration:
light:
  - platform: yeelight
    devices:
      <IP_ADDRESS>:
        name: Wall light
    custom_effects:
      - name: 'Fire Flicker'
        flow_params:
          count: 0
          transitions:
            - TemperatureTransition: [1900, 1000, 80]
            - TemperatureTransition: [1900, 2000, 60]
            - SleepTransition:       [1000]

It needs to be migrated to:
yeelight:
  devices:
    <IP_ADDRESS>:
      name: Wall light
  custom_effects:
    - name: 'Fire Flicker'
      flow_params:
        count: 0
        transitions:
          - TemperatureTransition: [1900, 1000, 80]
          - TemperatureTransition: [1900, 2000, 60]
          - SleepTransition:       [1000]

Just move everything to yeelight namespace, and remove 2 spaces vertical.
Description:
Moved yeelight into component to allow adding switches / sensors etc. Now only option to change yeelight power mode ( daylight / nightlight in yeelight ceiling lamps ), is to call service. Then its not possible to see current power mode. Added nightlight switch to turn on / off nightlight mode and get feedback about currently enabled power mode. I'm not sure about many things, regarding component -> platform communication and setup. I've looked at other components and tried to come up with something works. Whole config is identical as it was for light platform. Looking to early feedback if its done right, then I will add docs update if its ok.
Pull request in home-assistant.io with documentation (if applicable): https://github.com/home-assistant/home-assistant.io/pull/8845
Example entry for configuration.yaml (if applicable):
yeelight:
  devices:
    <IP_ADDRESS>:
      name: Yeelight

Checklist:

[X] The code change is tested and works locally.
[X] Local tests pass with tox. Your PR cannot be merged unless tests pass
[X] There is no commented out code in this PR.

If user exposed functionality or configuration variables are added/changed:

[X] Documentation added/updated in home-assistant.io

I am going to try this out later on, but would it make sense to convert yeelight to use their custom protocol alongside with this conversion? This way we would gain the unique ids for the devices, allowing renames in the UI: https://yeelight.readthedocs.io/en/latest/#usage

Thanks for initial feedback. I will work on tests and fix lint issues.
You mean id presented in discover_bulbs example ? Not sure where it should be used, I assume to somehow register devices with it ?
I was also wondering, how to approach light / switch synchonization of update. For example, when light is off, switching switch will turn it on, but then light waits up to 30s to update status. Is there any guideline how to synchonize between different platforms ? I was thinking light object could be passed to switch, but it looks like a bad coupling.

Yeah, I'm talking about using that id as unique_id inside homeassistant's entity registry (https://developers.home-assistant.io/docs/en/entity_registry_index.html). This will allow homeassistant to identify the bulb uniquely even if its IP address changes and will enable the "rename device" option in the UI (so no need to manually use friendly_name mappings). As this information is not available with mDNS discovery (what is currently used), it has not been possible earlier.
Wrt syncing, I think you can use the schedule_update (https://developers.home-assistant.io/docs/en/entity_index.html#subscribing-to-updates) even when polling is used, but I hope someone will confirm that or propose another approache.
Wrt coupling, I think it's fine to pass a reference of the light to the switch as they are coupled anyway.

I think I've figures out how to synchronize state between entities. Code taken from KNX integration.
Also I've put some fixes, regarding nightlight:
https://github.com/home-assistant/home-assistant/pull/21593/commits/d85bc75f19afd8d7a8c1e466647159849710954c#diff-6b4ac627753ee21cde162f000848ba3bR326
https://github.com/home-assistant/home-assistant/pull/21593/commits/d85bc75f19afd8d7a8c1e466647159849710954c#diff-6b4ac627753ee21cde162f000848ba3bR312

Fixed lint errors.
I've also asked on discord about help with writing tests. Response I've got was that requirements for test is not written in checklist, so maybe they are not needed ( https://developers.home-assistant.io/docs/en/creating_component_code_review.html ). I've tried looking at some similar platform, to get idea how to approach / what to tests, but still not sure how to write them correctly.
As for unique id, not sure how to approach that ? Should discovery_bulb be done in component setup ? Or register some background job, and when response come, update all bulb devices to put received id ? Also is it needed to be done here. Maybe putting too large PR will make it harder to merge and somehow blocks other PR regarding yeelight. There is ongoing effort to add ambilight support for ceiling 650. Also I was thinking about adding some more config options, in separated PRs. Or its better to make one big ?

I cannot say about the test requirements, but I had to create some when porting the tplink into a component.. I have not yet found time to test these changes and cannot currently promise when I'm able to do that, so I hope someone else chimes in.
Anyway, on the unique_ids and other improvements, I think it's better to done it in phases to keep it simple so please ignore that for now. That conversion will be a bit more involved, basically requiring creating a config entry flow (https://developers.home-assistant.io/docs/en/config_entries_index.html) to do the discovery and initializing the platforms based on the discovery results.

Moved services into component domain. I think this is ready.

@rytilahti Its ready, rebased on newest code. Can you take a look at it ? I've removed switch as I'm not sure about its UX ( What it should do on off etc). Will add sensor to get current working mode for daylight / nightlight for ceiling lights.

Could you fix the conflict? Thanks!

@syssi Thanks, rebased on new code. Are you planning on adding ceiling4 ambilight support ? I saw some post about it on community forum. I was planning to work on it, don't want to duplicate work. After merging this I wanted to make PR with python-yeelight bump and some cleaning, that will be possible with new lib version. And then I can work on ambilight support. Also I want to add power mode sensor, for better UX with moonlight mode for ceilings.

@zewelor I would be happy if you do most of the job. I don't own the device. I'd like to provide some support.

@MartinHjelmare PR with fixes here: https://github.com/home-assistant/home-assistant/pull/22347

@zewelor First, I really appreciate this effort which significantly improves usage of my yeelight 650. I've tested today in merged to dev branch. Yes - since now it allows me to control ambient lights as well. Great job, really! However there are several issues which I believe important to depict and possibly to fix maybe i next releases of this great contribution:

the most significant issue imo - if my configuration.yaml does not have any notions of yeelight platform it still discovered by hass (Bonjour/mDNS is here ;) for good) and before this contribution I had just auto-discovered yeelight light representing main bulb (yes, ambient lighs missed). With this contribution I have exception while hass load:

2019-03-27 20:12:15 ERROR (MainThread) [homeassistant.setup] Error during setup of component yeelight
Traceback (most recent call last):
  File "/Users/sam/0.homeassistant/home-assistant/homeassistant/setup.py", line 154, in _async_setup_component
    component.setup, hass, processed_config)  # type: ignore
  File "/usr/local/Cellar/python/3.7.0/Frameworks/Python.framework/Versions/3.7/lib/python3.7/concurrent/futures/thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/sam/0.homeassistant/home-assistant/homeassistant/components/yeelight/__init__.py", line 147, in setup
    conf = config[DOMAIN]
KeyError: 'yeelight'

and no lights at all.
So yes, after addition of yeelight: into configuration.yaml its autodiscovered my lights and now its shows two lights for device - main and ambient! great! as expected.
But the point is that overall user experience is completely suffered - instead of magical auto-discovered device which is really shows the power and elegance and magic of HA to end users - users would have to specify platform explicitly. Pitty. If hovewer KeyError whould be just suppressed in code it would provide significant improvement for user experience. Please consider. And yes - I do not specify my yeelight ip addresses - yes I want Bonjour and auto-discovery do it for me. Yes I do really dont want to modify my configuration.yaml for cases that do not introduce anything specific - I love when my new devices are auto-discovered ...
2. issue number two but less significant ):
which this contribution main light lost ability to change temperature and specify effect from UI:

So only brightness can be changed. Well, hmmm - presence of ability to control ambient light is really more important for me, but sounds as defect imo...
Thanks again for overall great contribution - hope you would share my vision on UX here ) Cheers!

Please open an issue if you suspect a bug.
If you want to suggest an enhancement please open a feature request in the Feature Requests section of our community forum.
Merged PRs should not be used for enhancement discussion or bug reports. If you've found a bug it's ok to make a review with inline comments and link to an issue that reports the bug.
Thanks!


Better error messages for mismatched types.

Example:
This: "xkcdpass -w 10"
Returns this: "No such file or directory (os error 2)"
But could return something like this: "Error: mismatched types. Try 'xkcdpass --help'."

I'll accept a PR that fixes this :)

Okay, I will work on this.

Weird... this is failing on --help or -h with the now error. Everything else works.

@pengowen123 You wrote some of the input code, so do you know what is going on here?

I added this to print the error in the unwrap_or_else closure:
println!("{}", e);
When I run xkcdpass --help with this change, it prints the help message. The closure looks like it should only be used for printing the help message. I don't think that is the right place to put the mismatched type error. It would be better to check the type of the IO error, then print a more useful message.


Modify coverages file

Dear developers,
I am starting to work with clear-cnv and I have one comment.
When I apply the matschores function using the coverages file obtained by workflow_reassignment I obtain the following error:
Exception: File needs to be formatted like: chr	start	end	gene	sample_0	sample_1	sample_n
I have realized that it is due to a missing column in the coverages file (the gene column).
So this error can be solved if the gene column is added, however, in my opinion manipulating files  can be error-prone. I suggest it would be very nice if the coverages file could incorporate the gene column.
Thanks for your work,
Eli

Hi Eli,
thank you very much for your advice. Beware that this "hack" only works if your data was generated on only one gene panel. If you have two or more bed files, then they get merged and the coverages are computed on this new big bed file, thereby they become unusable for CNV calling.
Please be aware that a panel re-assignment is usually not necessary if you have only one gene panel. Just start with the workflow_cnv_calling step and take a look at the QC files to see if your data is somewhat evenly distributed. ClearCNV's CNV calling is able to handle batch effects to a certain degree.
If you are certain that you need to separate batches even before you have seen the QC files, then you can indeed apply this little hack of re-using the coverages.
In any case, I really recommend to use workflow_cnv_calling instead of the single steps involved there. This also protects you from using data in the wrong spot.
I hope I understand correctly that you have only one bed file in your analysis but several batches. Otherwise I'd need some more feedback to find out what just happened there.
Cheers!
Vinzenz

Thank you Vinzenz!


Incorrect BRC-20 OP_RETURN transfer handling

Alex Labs pointed out a BRC-20 transaction which was indexed differently on Hiro’s indexer and BIS’s indexer.
The faulty transfer is https://mempool.space/tx/e90a43e1a126f41c790e135833ad89206a3e258467bda427bb169e596286096c
The faulty inscription is: b3b41d5e1588924b506ff9df02407d8685df6910d74dea1680747272f266d16ei0
In the transaction, this inscription should have been sent to OP_RETURN since it has 1 sats. But as Alex pointed out, Hiro’s indexer returned the sats to original owner. However there is no rule in BRC-20 which results in a return of funds if they’ve been sent to OP_RETURN. The only rule like this is if the funds are sent to miner as fee but it hasn’t happened in this transaction.
Due to this, sats balance of wallet bc1qkrewl9zclku2qngth52eezdyrwmjpcsppdkh6w is different on BIS & Unisat vs Hiro
https://bestinslot.xyz/bc1qkrewl9zclku2qngth52eezdyrwmjpcsppdkh6w#type=brc20
https://unisat.io/brc20?t=1698754465234&q=bc1qkrewl9zclku2qngth52eezdyrwmjpcsppdkh6w
https://ordinals.hiro.so/address/bc1qkrewl9zclku2qngth52eezdyrwmjpcsppdkh6w

Blocked by https://github.com/hirosystems/ordhook/issues/196

:tada: This issue has been resolved in version 2.0.0-beta.1 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:

:tada: This issue has been resolved in version 2.0.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


feat(ts/analyzer): Context aware reserved keyword handling

Implements AST validation for yield and await based on context (TS1212)
error[TS0]: TS1212 {
    span: Span {
        lo: BytePos(
            33,
        ),
        hi: BytePos(
            38,
        ),
        ctxt: #1,
    },
}
 --> a.ts:4:5
  |
4 |     yield;
  |     ^^^^^

error[TS2304]: NoSuchVar {
    span: Span {
        lo: BytePos(
            33,
        ),
        hi: BytePos(
            38,
        ),
        ctxt: #0,
    },
    name: yield#1,
}
 --> a.ts:4:5
  |
4 |     yield;
  |     ^^^^^

I'm sorry but I think this is the area of parser, and I think it's already implemented in the recent version of swc

Ah, no problem 👍


Fix error display for Json variant & add Custom variant

This PR fixes the Display trait implemented for Error and introduces a new variant Custom on the enum.
When implementing custom error handlers, the Display trait implemented for Error on Json variant
Error::Json((msg, status)) => write!(f, "{msg} (status: {status})")

makes it difficult for custom error implementation as the output of the above would look weird.
Here's an example:
let json_string = r#"{ "message": "test", "success": false }"#;
let err = worker::Error::Json((json_string.to_owned(), 500u16));
Response::error(err.to_string(), 500u16)

Outputs:
{ "message": "test", "success": false } (status: 500)

The above doesn't quite serve its purpose for the implemented Display trait.
It looks perfect when the Json variant is written so:
Error::Json((msg, _)) => write!(f, "{msg}")

Outputs:
{
	"message": "test",
	"success": false
}

So, is the point here that Display should output valid JSON, whereas Debug might include the status code?


feat: automatic row count

feat: #22657
Automatic Row Count
I added the "Automatic Row Count" option to Data Editor Preferences.

If the Automatic Row Count option is enabled, then the number of rows is calculated only once when the data viewer opens. Once it is calculated and the data is being fetched batch-wise, there will be no execution of the row count. But if you click on the refresh button, then it will show "200+, 400+" based on batch size.
https://github.com/dbeaver/dbeaver/assets/75378586/5908310d-e041-4a07-9911-09eadaea2183

Thank you for contribution

verified


Move train_cfg/test_cfg into model

LGTM, see if @hellock has any comments.

Kindly ping @hellock

Kindly ping @hellock

We also need to update documentation about configs, including the config examples.
Add deprecation notification in the documentation about configs.

We also need to update documentation about configs, including the config examples.
Add deprecation notification in the documentation about configs.

@xvjiarui  I tested all models using batch_test.py without checkpoint and find missed modifying the following configuration：
1 configs/faster_rcnn/faster_rcnn_r50_fpn_soft_nms_1x_coco.py
2 configs/faster_rcnn/faster_rcnn_r50_fpn_ohem_1x_coco.py
3 configs/lvis/mask_rcnn_r50_fpn_sample1e-3_mstrain_1x_lvis_v1.py
4 configs/lvis/mask_rcnn_r50_fpn_sample1e-3_mstrain_2x_lvis_v0.5.py

@xvjiarui  I tested all models using batch_test.py without checkpoint and find missed modifying the following configuration：
1 configs/faster_rcnn/faster_rcnn_r50_fpn_soft_nms_1x_coco.py
2 configs/faster_rcnn/faster_rcnn_r50_fpn_ohem_1x_coco.py
3 configs/lvis/mask_rcnn_r50_fpn_sample1e-3_mstrain_1x_lvis_v1.py
4 configs/lvis/mask_rcnn_r50_fpn_sample1e-3_mstrain_2x_lvis_v0.5.py


concurrent map read and map write error

Hi,
I'm trying to test the performance of Kuiper 1.2.0. I'm using a modified version of the edgex source which better suits my needs.
I'm testing 100 rules with each rule running 5 times every second. I get following error.
fatal error: concurrent map read and map write

goroutine 1722 [running]:
runtime.throw(0x130b210, 0x21)
        runtime/panic.go:1117 +0x72 fp=0xc002373768 sp=0xc002373738 pc=0x7c9d12
runtime.mapaccess2_faststr(0x12255a0, 0xc000ccc3c0, 0xc000d0d138, 0x4, 0x0, 0x0)
        runtime/map_faststr.go:116 +0x4a5 fp=0xc0023737d8 sp=0xc002373768 pc=0x7a6ec5
github.com/emqx/kuiper/xsql.Message.Value(0xc000ccc3c0, 0xc000d0d138, 0x4, 0x7f24fcd13330, 0x10, 0x18)
        github.com/emqx/kuiper/xsql/ast.go:676 +0xc5 fp=0xc002373860 sp=0xc0023737d8 pc=0xed03a5
github.com/emqx/kuiper/xsql.(*Tuple).Value(0xc002e245a0, 0xc000d0d138, 0x4, 0x0, 0x203000, 0x1)
        github.com/emqx/kuiper/xsql/ast.go:732 +0x45 fp=0xc0023738a0 sp=0xc002373860 pc=0xed0965
github.com/emqx/kuiper/xsql.multiValuer.Value(0xc00179d340, 0x4, 0x4, 0xc000d0d138, 0x4, 0xc0023739a0, 0x7a027b, 0xc003a$
        github.com/emqx/kuiper/xsql/ast.go:1119 +0x72 fp=0xc0023738f0 sp=0xc0023738a0 pc=0xed4912
github.com/emqx/kuiper/xsql.(*multiAggregateValuer).Value(0xc00179d380, 0xc000d0d138, 0x4, 0x7f24fcc74fd8, 0x110, 0x110)
        <autogenerated>:1 +0x5e fp=0xc002373940 sp=0xc0023738f0 pc=0xeff7de
github.com/emqx/kuiper/xsql.(*ValuerEval).Eval(0xc002373dd8, 0x141cb98, 0xc002373ae8, 0x6, 0xc003a5a628)
        github.com/emqx/kuiper/xsql/ast.go:1243 +0x889 fp=0xc002373a18 sp=0xc002373940 pc=0xed5929
github.com/emqx/kuiper/xstream/operators.project(0xc000d10f00, 0x2, 0x2, 0xc002373dd8, 0xc002e24500, 0x0, 0x12f2dca, 0x8)
        github.com/emqx/kuiper/xstream/operators/project_operator.go:107 +0x205 fp=0xc002373ba8 sp=0xc002373a18 pc=0x102$
github.com/emqx/kuiper/xstream/operators.(*ProjectOp).Apply(0xc00028b060, 0x1434fc8, 0xc000bfeaf0, 0x1258400, 0xc002fb9a$
        github.com/emqx/kuiper/xstream/operators/project_operator.go:48 +0x1dd9 fp=0xc002373e30 sp=0xc002373ba8 pc=0x102$
github.com/emqx/kuiper/xstream/nodes.(*UnaryOperator).doOp(0xc000c620a0, 0x1434fc8, 0xc000bfe4d0, 0xc000bfc060)
        github.com/emqx/kuiper/xstream/nodes/operations.go:109 +0x50c fp=0xc002373fc0 sp=0xc002373e30 pc=0xff7fac
runtime.goexit()
        runtime/asm_amd64.s:1371 +0x1 fp=0xc002373fc8 sp=0xc002373fc0 pc=0x801701
created by github.com/emqx/kuiper/xstream/nodes.(*UnaryOperator).Exec
        github.com/emqx/kuiper/xstream/nodes/operations.go:72 +0x1d3

It seems that the maps which cause the panic are the data and meta maps in the source code. To resolve this, before sending these maps over api.SourceTuple channel, I create a fresh copy of these maps, like following.

consumer <- api.NewDefaultSourceTuple(mapCopy(es.data), mapCopy(es.meta))

And my mapCopy function is defined as follows.
func mapCopy(m map[string]interface{}) map[string]interface{} {
	mn := make(map[string]interface{})
	for k,v := range m {
		if reflect.ValueOf(v).Kind() == reflect.Map {
			mn[k] = mapCopy(v.(map[string]interface{}))
		} else {
			mn[k] = v
		}
	}
	return mn
}

But the error still comes, is it a bug in the kuiper source or in my source plugin ?

@salmanazmat666 What do the rules look like? Could you please share an example? Thanks

SELECT meta(t1->name) as message, collect(meta(t1->origin))[-1]/1000.0 as origin, sum(t1) as st1 from stream1 GROUP BY SlidingWindow(ss, 30)

There are 100 such rules for values t1, t2, t3 etc.
And there are 100 streams, 1 stream for each rule.
Stream example are

create stream stream1 (t1 bigint) WITH (FORMAT="JSON", TYPE="edgex_custom")
create stream stream2 (t2 bigint) WITH (FORMAT="JSON", TYPE="edgex_custom")
create stream stream3 (t3 bigint) WITH (FORMAT="JSON", TYPE="edgex_custom")
... so on

The message is supposed to be immutabble. It is likely a problem of aliasing mechanism which try to modify the messages. Could you try to use rules which do no use alias?
SELECT meta(t1->name), collect(meta(t1->origin))[-1]/1000.0, sum(t1) from stream1 GROUP BY SlidingWindow(ss, 30)

So the issue has disappeared after removing the aliases. But aliases are very important for formatting of the message, otherwise the message is not much readable, for example I get following result of the above mentioned rule.
[{"kuiper_field_0":1624338035800401,"meta":"t1","sum":596}]
This is not readable at all and not even deterministic. Can you please take this issue as a bug ?

@salmanazmat666
Thanks for verifying. I agree this is a bug. The alias already have several known issues and we need to look into them together as a whole and we are working on them.
Currently, you can use DataTemplate to format the output including rename those fields.

Hello @ngjaying ,
I have the same exception with 53 rules on the same stream... the exception raises usually randomly.
I have a serious problem in production beacause kuiper crashs all the time... :-( but if I delete the edgex core data events kuiper restarts successfully.
Your solution doesn't work for me because I have this SQL:
 "sql": "SELECT (max(AI3) - min(AI3)), collect(*)[1]->AI3, meta(id), meta(Origin) FROM edgex_events GROUP BY COUNTWINDOW(2, 1) FILTER(  WHERE meta(device) = \"TAW_XP_4\" AND AI3 != nil )  HAVING (max(AI3) - min(AI3)) > 1000"
and I have this result:
 {
      "kuiper_field_0": 2000,
      "kuiper_field_1": 25000,
      "meta": "85c93b82-1996-4958-a0bf-7684a8458df9",
      "id": 5
    },

and I cannot distinguish between meta(id) and meta(Origin) and it overwrites both with meta...
if I use alias I have not this problem...
So I need to understand which edgex events or data cause this exception and I tried to change kuiper configuration with
debug: "true" but after 36 hours the server drive ran out of space beacause there were two files one of 4GB (backup) and one of 2GB (current stream.log), the free space is 6GB at startup.
Then I tried to change following the guide here
This is docker-compose entry in edgex foundry environment:
rulesengine:
    container_name: edgex-kuiper
    depends_on:
    - app-service-rules
    environment:
      EDGEX__DEFAULT__PORT: 5566
      EDGEX__DEFAULT__PROTOCOL: tcp
      EDGEX__DEFAULT__SERVER: edgex-app-service-configurable-rules
      EDGEX__DEFAULT__SERVICESERVER: http://edgex-core-data:48080
      EDGEX__DEFAULT__TOPIC: events
      KUIPER__BASIC__CONSOLELOG: "true"
      KUIPER__BASIC__RESTPORT: 48075
      KUIPER__BASIC__DEBUG: "true"
      KUIPER__BASIC__ROTATETIME: "1"
      KUIPER__BASIC__MAXAGE: "1"
    hostname: edgex-kuiper
    image: emqx/kuiper:1.2.0-alpine
    volumes:
      - kuiper-data:/kuiper/data:z
    networks:
      edgex-network: {}
    ports:
    - <IP_ADDRESS>:20498:20498/tcp
    - <IP_ADDRESS>:48075:48075/tcp

This is kuiper.yaml
/kuiper/etc $ more kuiper.yaml
basic:
  consoleLog: true
  debug: true
  fileLog: true
  ip: <IP_ADDRESS>
  maxAge: 72
  maxage: 1
  pluginHosts: https://www.emqx.io/downloads
  port: 20498
  prometheus: false
  prometheusPort: 20499
  restIp: <IP_ADDRESS>
  restPort: 48075
  rotateTime: 24
  rotatetime: 1
rule:
  checkpointInterval: 300000
  qos: 0
  sendError: true
sink:
  cacheThreshold: 10
  cacheTriggerCount: 15
  disableCache: true

The maxAge and rotateTime did not overwrite!
What did I do wrong?
Any suggestion is appreciated
Best regards
Ray

Hi Ray,
Regarding the config setting, this looks like a bug. As a workaround, you can edit the kuiper.yaml directly and restart the container instead of setting it be env.
For the alias problem, it should be fixed in the next release, we have a bet build lfedge/ekuiper:1.3.0-beta.0-alpine. However, the version will only supports EdgeX v2, I am wondering if you would like to upgrade.

Thanks @ngjaying
I resolved by external volume
    volumes:
      - kuiper-data:/kuiper/data:z
      - ./res/edgex-kuiper/kuiper.yaml:/kuiper/etc/kuiper.yaml:z


[SWADE] NPCs with 0 max wounds show estimation

In SWADE it is very common to have NPCs with 0 max wounds (wounds can be compared to HP but reversed (more wounds is less HP)). The problem is however, that HE does not show the estimation above the token when the actor has 0 max Wounds. This lets players know immediately, that they are dealing with an actor that has 0 max wounds (which is bad because NPCs can have more that 0 max wounds).
Changing the swade.js to this seems to work for me but I haven't looked at the full code so I can't really tell if this has the potential to break anything. In my limited testing it seems to work fine though.

I'm not sure if that is common for you for most GMs of the system. As in, "will someone open an issue asking me to reverse the change if I accept this?"
It's just a strange request, you'd have to set up the wounds if the players were to hit the NPC anyway, right?
I'd rather this become a general setting.


Don't simplify filter with custom functions over projection

Don't simplify filter with custom functions over projection to avoid suboptimal SQL plans and allow window functions (these function not allowed in WHERE clause)
TODO add more detailed description after tests are green

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: kosinsky sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Thanks @kosinsky. Sorry for the long delay.

+1 thanks for this @kosinsky!


build local, deploy afterwards

Hi,
do you have any plans to implement a feature to prepare the the build locally and deploy the final build to the servers? Basically what already existed with build_script and copy_strategy for version 2.x.
Having all kind of deployment relevant software installed on every server and building the same .jar file on a dozen of servers doesn't feel right.
Best regards,
Daniel

No plans, but largely because it's not necessary anymore, for two reasons:

We do have plans to make loading the SCM integration optional, when that's done, you can skip loading the Git deploy recipe, for example
Largely, since SSHKit was extracted, it's trivial to do something such as the following: http://lee.hambley.name/2013/06/11/using-capistrano-v3-with-chef.html

This refers to using Chef, but essentially anything where you build something locally, upload it to a server, and do something with it would work similarly.

@dfreudenberger Take a look at https://github.com/wercker/capistrano-scm-copy. It might be what you're looking for.

https://github.com/Bladrak/capistrano-rsync works well for me.

As @leehambley mentioned, we don't have plans to implement a build-push system, but we are working to make the SCM implementation more pluggable (and completely optional).
There are also gems available that seem to address your scenario (see previous comments).
I'll close this issue since it is not something Capistrano itself will be addressing directly.


Feature: Support fetching artifacts from Canner Enterprise

Description
In order to make VulcanSQL could integrating with Canner Enterprise, in this PR, we need to achieve two goal:

support CannerPersistenceStore to read artifacts by each workspace from Canner Enterprise Storage and merge them into one artifact for Canner Enterprise automatically.

support CannerProfileReader to generate Canner Enterprise Data Source Connector used profiles, it will connect to Canner Enterprise Storage and get the workspaces information using Vulcan for producing the profiles info. (Do not create profile.yaml and then read it )

Solution
Create the CannerPersistenceStorage to get the artifact's newest result.json and workspaceSqlName by the below steps:

Reading the master value in the indicator.json to know the deployment folder name <timestampe_hash> by traversing each /000000-0000-0000-00000/<workspaceId>/vulcansql folders

Get the artifact in the /<timestampe_hash> path on the MINIO.

Get the workspaceSqlName in the indicator.json

Read each artifact result.json and add the prefix workspaceSqlName to the soruceName, urlPath, templateSource in the schemas, templates, and specs and then merge all result.json to the one-for-one VulcanSQL Pod could run it:

{
  "templates": {
    "eason_ws/artist/artist": "...",
    "freda_ws/artist/works": "..."
  },
  "schemas": [
    {
      "sourceName": "eason_ws/artist/artist",
      "urlPath": "eason_ws/artist/:id",
      ...
      "profiles": ["profile-eason_ws"]
      "templateSource": "eason_ws/artist/artist",
    },
    {
      "sourceName": "freda_ws/artist/works",
      "urlPath": "freda_ws/artist/:id/works",
      ...,
      "profiles": ["profile-freda_ws"]
      "templateSource": "eason_ws/artist/works",
    }
  ],
  "specs": {
    ...
  }
}

Create the CannerProfileReader to connect Canner Enterprise Storage, and do the below steps:

Read the workspace Id and SqlName from each workspace including vulcansql subfolder which means using the VulcanSQL Analysitic API service.
Read the Canner Enterprise connector needed connection info from environment variables. ( Please see the README )
Create every profile with the same connection info and provide the name to profile-{workspaceSqlName} ( help us to let artifacts could find the profiles  infomation ) and database field of the connection options to the workspaceSqlName  ( We create the profiles directly to use, not create profiles.yaml and then read it. )

The reason we create profiles information is that we could not only one Canner Enterprise connector profile for all different workspaces analytics API used, because the profile connected to Canner Enterprise by PG Wire Protocol needs to give a workspace SQL name ( set in database field ), but one Canner Enterprise connector profile only could set one database field.
In order to get the above artifact files and worksapceSqlName from MINIO, we need to use the @canner/canner-storage, but it’s the private package, so we should have a place to set the NPM token, otherwise using the CannerPersistenceStorage be our artifact reader source, we should support a new type to recognize through provider field in artifacts options in vulcan.yaml:
# vulcan.yaml
...
artifact:
  provider: Canner
  serializer: JSON
  filePath: <rootPath> #<PHONE_NUMBER>-0000-000000

extensions:
  canner-store: '@vulcan-sql/extension-store-canner'

profiles:
  - type: Canner

For using the CannerProfileReader, we need to set the type be Canner in the profiles like above.
In order to use the @canner/canner-storage to connect the Canner Enterprise Storage, and make the CannerProfileReader could generate the profiles information with Canner Enterprise connector's connection, we also need to set the environment variables, please see the README
Issue ticket number
closes #170
Additional Context

Change the fs.rmDirSync to fs.rmSync for BigQuery and Snowflake test cases to make sure the test directory could force remove, because fs.rmSync could support force remove.

Codecov Report
Patch coverage: 99.11% and project coverage change: +0.17 :tada:

Comparison is base (2b5e282) 91.12% compared to head (facc75c) 91.29%.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #180      +/-   ##
===========================================
+ Coverage    91.12%   91.29%   +0.17%
===========================================
  Files          316      321       +5
  Lines         5091     5204     +113
  Branches       679      688       +9
===========================================
+ Hits          4639     4751     +112
  Misses         321      321
- Partials       131      132       +1

Flag
Coverage Δ

build
90.50% <ø> (ø)

catalog-server
100.00% <ø> (ø)

cli
81.48% <ø> (ø)

core
94.18% <ø> (ø)

extension-dbt
97.43% <ø> (ø)

extension-debug-tools
98.11% <ø> (ø)

extension-driver-bq
85.41% <ø> (ø)

extension-driver-duckdb
96.61% <ø> (ø)

extension-driver-pg
96.11% <ø> (ø)

extension-driver-snowflake
96.26% <ø> (ø)

extension-store-canner
99.11% <99.11%> (?)

integration-testing
90.27% <ø> (ø)

serve
86.99% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

packages/extension-store-canner/src/lib/config.ts
80.00% <80.00%> (ø)

...on-store-canner/src/lib/canner/persistenceStore.ts
100.00% <100.00%> (ø)

...nsion-store-canner/src/lib/canner/profileReader.ts
100.00% <100.00%> (ø)

...ges/extension-store-canner/src/lib/canner/utils.ts
100.00% <100.00%> (ø)

...s/extension-store-canner/src/lib/storageService.ts
100.00% <100.00%> (ø)

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


remove rejectUnauthorized parameter in api client instance

can we get rid of it here? I know what's for, but since we have no problems with certificates and connection between layers - we can go without it.
Originally posted by @mkucmus in https://github.com/DivanteLtd/shopware-pwa/pull/743

For additional context


Fixed issues #100, #88, and #98

Fixed soundness bug in encoding of super-class fields.
Added a counter for new allocations. Introduces a global variable heapCounter in jayhorn.utils.HeapCounterTransformer that gets incremented every time an object gets allocated. The reference is then assigned to the current value of the counter. I.e.:

x = new A

becomes
heapCounter++
x = heapCounter

and then some stuff about the type. Using this counter fixes issue #88 and allows us to drop the assume x!=null after the new.

fixed a few other bugs and removed some junk.
added some nicer printing of results for cases where the proof is successful.

Coverage decreased (-0.5%) to 62.421% when pulling c1671124b63374986598b70a9f3ddad9b7dce43f on caller_id into f3f8954bd810b57b40aee3857e497a655220101d on devel.


Document versioning?

I am little bit lost here. Is this document versioning? How do enable/view on the portal? Please clarify?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fdcc897a-d6df-3305-e9cb-01889a3c5d4f
Version Independent ID: 739a9fc7-3782-a860-ac6d-4194fd1ce2e8
Content: Working with the change feed support in Azure Cosmos DB
Content Source: articles/cosmos-db/change-feed.md
Service: cosmos-db
GitHub Login: @TheovanKraay
Microsoft Alias: thvankra

@manish-jain-1 Please go through below links to get information regarding the Change feed. if you still have a question, please get back to us:
https://youtu.be/sYkKXSTws4E
https://azure.microsoft.com/en-us/resources/videos/azure-cosmosdb-change-feed/
Hope this helps

@manish-jain-1 Please let us know if we can help with this further or we can go ahead and close the thread.

@manish-jain-1 We will proceed to close this issue now.
If there are further questions regarding this matter, please comment and we will gladly continue the discussion.


Resolve incompatibility with latest selenium-webdriver

selenium-webdriver v2.52.0 was unable to automatically detect the local ChromeDriver instance I had on the PATH in my development environment. Rather than debug that incompatibility, I locked down the version to v2.52.0 via a npm-shrinkwrap.json.
Given more time, I should get to the root of the issue and file a bug against selenium-webdriver if it turns out to be an issue on their end.

For the record, the specific error I get is
     TypeError: Path must be a string. Received -1
      at assertPath (path.js:7:11)
      at Object.join (path.js:1213:7)
      at Object.exports.findInPath.opt_checkCwd [as findInPath] (node_modules/selenium-webdriver/io/index.js:232:28)
      at new ServiceBuilder (node_modules/selenium-webdriver/chrome.js:183:29)
      at getDefaultService (node_modules/selenium-webdriver/chrome.js:362:22)
      at Driver (node_modules/selenium-webdriver/chrome.js:771:34)
      at Builder.build (node_modules/selenium-webdriver/builder.js:464:16)
      at ChromeWebDriverBrowser.getSeleniumDriver (node_modules/sw-testing-helpers/node/webdriver-browser/web-driver-browser.js:161:8)
      at Context.<anonymous> (automated-suite.js:72:24)


Why MergeTree  need Data column?

create table SQL:
CREATE TABLE events_local
(
    src_ip UInt32,
    dest_ip UInt32,
    src_port UInt16,
    dest_port UInt16,
    req_time UInt64,
    page_urlhost String,
    page_urlpath String,
    page_full_url String,
    resp_size UInt64,
    content_type String
) ENGINE = MergeTree(intHash64(src_ip), intHash64(req_time), (dest_ip, src_port, dest_port, req_time, page_urlhost, page_urlpath, page_full_url, resp_size, content_type, src_ip), 8192)

I can't create success. error log is need Data&primary key column.
I can't find any about primary key info,This is table don't need Data type column.
How to fixed this problem.

You need Date (not Data, it is Date, i.e. day, '2017-03-30' for instance), because MergeTree engine splits data into partitions by months, and the engine obviously needs a way to know what date a record belongs to and it only was taught to use Date typed field for the purpose.
So, you need a Date typed field in  table, like
CREATE TABLE testTables (
    date Date,
    value String
) ENGINE=MergeTree(date, (value), 8192);

FYI You can create dummy date filed (which you will not use into INSERT and SELECT queries).
In that case all table's data belong to the same partition (with name 197001), but such commands as FREEZE PARTITION, DROP PARTITION, etc does not make sense.

so,If i create dummy date,Data is not balance in every node.


Go-To-Definition Jump to Empty File, with Extra Tag File Set

I added an extra tag file, in which contains tags with relative file path, for atom-ctags. When I triggered go to definition, the atom open an empty file, whose file name is indicated by tags.
This phenomenon is due to file path issue. Because the directory where the extra tag file located is different from the atom working directory.
I think we could set the file path I set for extra tag file as a prefix to the indexed relative file path in external tag file, when performing a go to definition.

:worried: Hello, same effect in multi project workspace:
I have a several projects in workspace, and all of them contains .tags file, and that file contains generated records:
ComparisonTable mvid/classes/ComparisonTable.js /^   * @class mvideo.Class.ComparisonTable$/;"  c

The issue is in that when go-to-definition command has invoked an empty file has been opened, and that part of code tries to open it by providing relative path to the method:


[Azure] Add sovereign cloud support for disks module

The Base URI is incorrectly resolved for the disks client, and should be fixed to read from the environment.
Addresses #811

@yorinasub17 would you please close it, addressed in this merged PR #884

Closing as done in #884


Feat/nonce buckets against replay attacks

Goal
The goal of this PR is to implement and use a replay attack prevention technique in two extrinsics that take externally created signatures of externally provided data:

create_sponsored_account_with_delegation
add_provider_to_msa

The replay attack prevention provides an early exit for both of these extrinsics.
Related to #327
Discussion
Sets up a system of rotating virtual buckets to store signatures, using the bucket number and signature instead of a nonce.  Signatures must provide an expiration block. This is used to compute the bucket number it's expected to be in, and find out whether the signature has been recently submitted.
Signatures are rejected if they are found in the registry or the associated payload has an expiration too far in the future (> MortalityBucketSize * NumberOfBuckets)
Every MortalityBucketSize * NumberOfBuckets blocks, the bucket rotation starts over, the old signatures are cleared, and the new signature is added.
To be able to check whether a virtual bucket needs to be cleared, bucket numbers are mapped to a mortality block.
Signature storage is really a StorageDoubleMap where K1 = bucket number and K2 is the signature.  The value stored is the expiration provided in add_provider_payload (currently unused).  The bucket expiration block, being the first key in the StorageDoubleMap  makes it extremely easy to "clear" a virtual bucket using drain_prefix (via IterableStorageDoubleMap trait).
Checklist

[x] Refactor EXPIRATION_BLOCK_VALIDITY_GAP?
[x] Set up genesis for buckets in mocks
[x] New Errors, Constants in pallet
[x] Values in mock
[x] Values in frequency and frequency-rococo runtimes
[x] StorageMap for the bucket --> mortality map
[x] StorageDoubleMap for BucketSignature --> mortality
[x] Helpers in pallet
[x] Unit tests for helper fns
[x] Replay tests passing
[ ] Chain spec updated
[ ] Updated the js/api-augment code if a custom RPC added/changed
[ ] Design doc(s) updated
[ ] Tests added
[ ] Benchmarks added
[ ] Weights updated

/run_benchmarks

Fantastic! 👍 You made something so difficult look so easy.


Problem with Join

I seem to having problems using Promise.join.  I have the following code:
var Promise = require("bluebird");
var Request = require('request');

function callhttp(host, cb) {

    var cert = " ";

    var options = {
        url: 'https://' + host
    };

    function callback(error, response, body) {
        if (!error && response.statusCode == 200) {
            var cert = response.connection.getPeerCertificate().fingerprint;
            cb(cert);
        }
    }
    Request(options, callback);
}

callhttp("www.google.com", function(_cert) {
  //  console.log(_cert);
});

var join = Promise.join;
    join(callhttp("www.google.com", function(_cert) {
        console.log(_cert);
    }),
    callhttp("www.microsoft.com", function(_cert) {
        console.log(_cert);
    }),
    callhttp("www.adobe.com", function(_cert) {
        console.log(_cert);
    }),
    function(first, second, third) {
        console.log({fp1: first, fp2: second, fp3: third});
    });

However the join does not seem to work as documented and outputs this:
{ fp1: undefined, fp2: undefined, fp3: undefined }
E9:E2:F5:B9:CD:DA:73:CD:41:94:20:EA:51:88:4D:F0:FC:11:6F:4E
94:C1:5B:66:01:00:B0:23:E0:C7:DA:A9:AC:47:3F:BC:3B:E1:C0:53
09:BF:C6:28:E9:E4:94:A7:2B:CD:EE:2B:86:AB:44:A3:72:39:28:C2

callhttp doesn't return a promise (or anything) for one, so you're just joining undefined values and passing them to your handler.
https://github.com/petkaantonov/bluebird/blob/master/API.md#promisification

There is a couple of basics that are wrong, I am actually working on a beginner's guide so stay tuned.
In the meanwhile...
You never want to use callbacks manually anymore, as wiring callbacks manually to promises effectively results in code that has all the problems of callbacks with some additional verbosity.
So first lesson is to promisify at the lowest level:
var Promise = require("bluebird");
var Request = Promise.promisify(require('request'));

Now Request is a function that returns a promise (and doesn't take a callback anymore).
Secondly, callhttp is not returning a promise - promises are wired together with the return statement. It also should not take any callback since it's using promises now.
function callhttp(host) {

    var cert = " ";

    var options = {
        url: 'https://' + host
    };

    return Request(options).spread(function(response, body) {
        if (response.statusCode == 200) {
            return response.connection.getPeerCertificate().fingerprint;
        } else {
            // Just an example, 200 is not the only successful code
            throw new Error("HTTP Error: " + response.statusCode);
        }
    });
}

As for join, the signature is promises..., handler so:
var join = Promise.join;
join(
    callhttp("www.google.com"),
    callhttp("www.microsoft.com"),
    callhttp("www.adobe.com"),
    function(first, second, third) {
    console.log({fp1: first, fp2: second, fp3: third});
 });

Cool - thank you very much. That did the trick.


[CAMEL-15188] Add proxy type query parameter to camel-telegram component, add support for SOCKS proxy

Added support of SOCKS proxy to telegram component by adding proxyType query parameter.
Nothing fancy, just passing proxyServerType to HTTPClientPolicy.
Default value for proxyType is HTTP, that ensures existing configuration will not be touched by this patch.

Camel 2.x is only for bug fixes etc so this should be on master branch

Thanks has merged this in camel-2.25.x branch (only active). You are welcome to work on the PR for the master branch

Since it's a new feature I thought it would be better to have in 2.x if we eventually release 2.26.0. No problem anyway.

Its essentially something that makes camel-telegram useable in russia (need that proxy). We wont do a 2.26 so this is the only way.


See if it can be made to work on 64 bit systems (without segfaulting)

Sorry for a very late response.
Since the project is no longer maintained (see README), I will not attempt to find/fix the problem, but would link to a better fork if one comes up.
Sorry I wasn't be of much help.


Fix error on ReturnCommandParameterProperty

CommandParameter is always NULL.

Hi contrix09 when you'll integrate the fix in NuGet package? Thanks!


How does it work in view/form

Great gem.
When I tried it from my rails console, everything works fine. But when I implement to my view or controller I get error: undefined method image_data_uri= for nil:NilClass
This is how I did it:
Form view:
= f.text_field :image_data_uri
Saving only from controller without controller
before_action :save_data_uri, only: [:create]
def save_data_uri
      @album.image_data_uri = "data:image/gif;base64,R0lGODlhAQABAJEAAAAAAP////8AAP///yH5BAEAAAMALAAAAAABAAEAAAICVAEAOw=="
      @album.save
end

I need to do it from my view and looking for a solution for my form.
Hope you can help me figure it out.

Hi @Rubioli
It seems that you didn't set @album variable, that's why you got this error.

Thanks @timsly


Fix warnings & filename typo.

Thanks for merging my changes.  Here's a few more updates.

Looks good.


Add deuterium, tritium and boron species

D and T have different charge-exchange cross-sections, and so are now treated as different species by radas. This means that we can no longer set a unique value of atomic number for an AtomicSpecies, and instead need to handle their atomic number manually.

Is there a corresponding radas change that we can link here?

@hassec  this came in during this PR https://github.com/cfs-energy/radas/pull/18


Do not work properly with django orm

Hi, @saileshkush95! It would be helpful to provide more information about your use case and any steps to reproduce the issue. At the moment it's a little unclear to me what the issue is


SSO/w JWT Tokens not redirecting after upgrade to Spring Boot 2

Summary
After upgrading our authentication server from Spring Boot 1.5.13 to 2.1.3 it stopped redirecting after successful logins.   Spring Security 5 tries to reload the /login page again after successfully loging in instead of allowing the request to go back to the protected resource behind the proxy server as it does with Spring Boot 1.5.13.
It is my understanding that the root cause is that only the oauth2 client has been ported from its Spring Security extension codebase to Spring Security Oauth2 and that in the meantime the authentication server must build the JWT token and put it in the Authorization header. I did that by adding custom filters, but the redirect after successful login still does not work.
Actual Behavior

User tries to access the Angular site via an SSO-enabled proxy server.
Spring Security on the proxy server redirects the user's browser to the Login page on the authentication server.
The user signs in and posts the Login form to the authentication server.
The authentication server forwards the user back to the Login page.

Expected Behavior

User tries to access the Angular site via an SSO-enabled proxy server.
Spring Security on the proxy server redirects the user's browser to the Login page on the authentication server.
The user signs in and posts the Login form to the authentication server.
The authentication server creates a JWT oauth2 token, puts it in the Authorization header, and then forwards the request to the original target URI, in this case, the Angular Example site.

Configuration
The auth server configuration can be found https://github.com/smitchell/spring-security-5-upgrade_sso-auth-server. The proxy server uses EnableOAuth2Sso. I added custom filters to create and verify JWT tokens using this example: https://github.com/keysh/jwt-security.
Version
Spring Boot 2.1.3.RELEASE.
Sample

Angular Example - If you bypass the proxy and go to the angular example route the page loads. TRY IT.
Authentication Example - If you bypass the proxy and go to the authentication server root you can sign in and access a protected page. TRY IT
Zuul Proxy Example - If you try to access the angular example via the proxy, then you do not get forwarded to the angular example after logging in. TRY IT

All source code is in Github:

Angular example Github: https://github.com/smitchell/cloud-foundry-angular-example
Proxy Server Github: https://github.com/smitchell/spring-boot-netflix-zuul-proxy
Auth Server Github: https://github.com/smitchell/spring-security-5-upgrade_sso-auth-server

yes,I want to konw too

Any updates?

It is my understanding that only the OAuth2 client has been ported from the OAuth2 extension to Spring Security core. The rest is being ported now, but I don’t what release. We opted to leave are authentication server at Spring Security 4 until they finish. It works fine with the Spring Security 5 OAuth2 clients running on all our resource servers.

@smitchell I started looking into this for you but startup failed for spring-boot-netflix-zuul-proxy. I'd prefer to run this locally (instead of PCF) for debugging purposes. Can you please update the app/config(s) to ensure it starts up locally.
Also, I wasn't able to run cloud-foundry-angular-example as well. Please provide detail instructions on what I need to do to get that up and running as well. Looks like I might need to install some Angular components?

It is my understanding that only the OAuth2 client has been ported from the OAuth2 extension to Spring Security core

Spring Security 5.1 have both client and resource server features and with the soon to be releases 5.2, both client and resource server features will be at feature parity with Spring Security OAuth legacy features. We are targeting 5.3 for initial release of Authorization Server.

@jgrandja - I did this on a Linux machine and had all kinds of difficulty with pcfdev, which is why I fell back to PWS.

I will review the PWS instructions. I'm guessing the problem is the routes, which must be changed.
I will try again to get it to work with pcfdev, but this time, on my Mac. The problem I had was with pcfdev Cloud Services. The service registration did not work.

I will let you know how it goes.

@jgrandja - I updated the readme, application.yml, and manifest files on GitHub (projecte links are above):

Deploy all three projects with the routes commented out so new routes get mapped.
Update the application.yml and manifests with the routes that were mapped per instructions on README page.

Re-push the apps.
Follow the testing instructions.

The only glitch is that after a successful login to [proxy route]/angular-example/, you get a permission denied on the second reload of the /login page. That still shows the bug. It should have redirected you to [proxy route]/angular-example/ after the successful login. I'll mess around with pcfdev on my Mac next.

@jbiblio - I can't run pcfdev. cf 6.46 says that pcfdev-pas.v.<IP_ADDRESS>-build.0.0.71-darwin.tgz is not compatible. I don't know what I'm supposed to do with pcfdev_2.1_osl.txt. pcfdev on Linux: fail. pcfdev on Mac: fail. I don't have a Windows machine. If you have have a working pcfdev environment then you should be able to follow my instructions above there too.

@smitchell Maybe I wasn't clear in my last comment? My preference is to run the 3 apps on my locally machine - NOT in PCF. It's easier for me to debug and troubleshoot and it takes environment related complexities out of the equation.
So if you can update the app configs so it works in a local setup that would be helpful.

@jgrandja - please follow the updates i made to the readme files yesterday.

@smitchell I just cloned spring-boot-netflix-zuul-proxy and when I run Application I get the following error:
2019-08-22 15:13:58.933 ERROR 79878 --- [           main] c.n.d.s.t.d.RedirectingEurekaHttpClient  : Request execution error. endpoint=DefaultEndpoint{ serviceUrl='https://eureka-6e9cd6de-58b7-4b08-a10e-5e176a7a7cea.cfapps.io:8761/eureka/}

What I'm looking for is an easy/quick setup to a point where I can reproduce the issue you are having. I do not want to make application configuration changes to make this work locally on my laptop. Your sample and instructions are based on deploying to PCF which is what I'm trying to avoid here.
It would be a lot more efficient if you can setup these 3 applications (possibly in separate branch) so they startup on my local machine. This will allow me to be the most effective in troubleshooting. I will wait until you're able to provide this.

@smitchell I'm not sure if there is a PCF distribution that can be installed locally? Regardless if it's possible, I do not want to run these in a PCF env. I simply want to start the 2 Spring Boot apps - auth server and proxy - and the Angular app.

Got it. I'll work on that.
On Fri, Aug 23, 2019 at 9:30 AM Joe Grandja<EMAIL_ADDRESS>wrote:

@smitchell https://github.com/smitchell I'm not sure if there is a PCF
distribution that can be installed locally? Regardless if it's possible, I
do not want to run these in a PCF env. I simply want to start the 2
Spring Boot apps - auth server and proxy - and the Angular app.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/spring-projects/spring-security-oauth/issues/1676?email_source=notifications&email_token=AACX4E2QHH2Z7LXWWFAN3K3QF7YBFA5CNFSM4HMLFOD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5AMCBI#issuecomment-524337413,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AACX4E5QZ2L6X633RTZGM3DQF7YBFANCNFSM4HMLFODQ
.

@jgrandja - I committed changes to the projects to run on localhost:4200, 4201 and 4202. I could use your help in understanding why that changed the behavior of Spring Security. Going through the proxy (http://localhost:4201/angular-example/) it now shows the 3-legged OAuth2 dialog.

@smitchell I was able to start up proxy and auth-server on my local machine but SSO did not work.
I first tried accessing the proxy via http://localhost:4201 and this triggered the SSO (authorization_code) flow with the auth-server on http://localhost:4202. After I authenticate with the auth-server, the browser gets redirected back to the proxy with the code but the oauth-client fails to complete the Authorization  Code grant flow. The reason for the failure is because it cannot correlate the previous Authorization Request with the incoming Authorization Response containing the code. This is because the Cookie name used for both the auth-server session and proxy session is the same UISESSION:
server:
  servlet:
    session:
      cookie:
        name: UISESSION

So essentially what happens here is that the auth-server session cookie overwrites the previously generated proxy session cookie with the same name UISESSION.
After I make the following change to the auth-server application.yml:
server:
  servlet:
    session:
      cookie:
        name: PROVIDER

it works as expected. This may be related to the issue you are having in PCF?

@jgrandja - I'm not having the same experience. I updated the application.yml, but instead of going to the angular site after login I am presented with the 3-legged oauth page "Do you authorize "zuul-proxy-example" to access your protected resources?" I'm going to update to 2.18-RELEASE and retest.

@jgrandja - After a lot of testing this weekend I understand what is happening, but I don't know why. After you sign-in, the user is redirected to the proxy /login page instead of /angular-example. That is why I get the "Do you authorize "zuul-proxy-example" page.
Setup

Clone https://github.com/smitchell/spring-security-5-upgrade_sso-auth-server
mvn spring-boot:run
Clone https://github.com/smitchell/cloud-foundry-angular-example
mvn spring-boot:run
Clone https://github.com/smitchell/cloud-foundry-angular-example
ng serve  --baseHref=/angular-example/

Steps to Reproduce:

Navigate to http://localhost:8085/angular-example/
Login as "user" / "password"
The third-party auth page appears. Click Agree.
The Angular home page is displayed.

Reason for third-party auth page appearing
To see the unnecessary redirect to proxy /login page (http://localhost:8085/login) remove "http://localhost:8085/login" from the registered redirects by commenting out line 25 and uncommenting line 26 in AuthClientDetailsService.java. Restart the Auth server and try the steps above. You now get this message:
error="invalid_grant", error_description="Invalid redirect: http://localhost:8085/login does not match one of the registered values."
I am back to my original question in this Issue, why does using @EnableOAuth2Sso on a Zuul proxy, not redirect the user back to the originally requested URI after successfully logging in?

@jgrandja - I apologize for not catching this in my previous comment (see below). I assume this means that the problem is in the proxy configuration since it passes a redirect URL of "&redirect_uri=http://localhost:8085/login" instead of "&redirect_uri=http://localhost:8085/angular-example." Is that correct, and if so, why isn't /angular-example being passed?

GET http://localhost:8085/angular-example/ 302
GET Location: http://localhost:8085/login 302
GET Location: http://localhost:8084/oauth/authorize?client_id=zuul-proxy-example&redirect_uri=http://localhost:8085/login&response_type=code&state=pgAptw 302

GET http://localhost:8084/login 200
POST http://localhost:8084/login 302
GOT Request URL: http://localhost:8084/oauth/authorize?client_id=zuul-proxy-example&redirect_uri=http://localhost:8085/login&response_type=code&state=aFcXyH 200
POST http://localhost:8084/oauth/authorize
GET http://localhost:8085/login?code=9jlmH7&state=aFcXyH 302
GET http://localhost:8085/angular-example/ 200

@jgrandja - In Prod, with a Spring Security 5 client on the Zuul proxy calling a Spring Security 4 Authentication Server, there is 301 on the proxy login page that is missing when I run my code locally. Is it because of Cloudflare and not related? The extra 301 is returned for https://hostname/login before and after the custom login page. Note that on step 9, the proxy server redirects to the originally requested URI. That is what is missing after upgrading the auth server to Spring Security 5.


Broken DeleteField semantics on Spark

Deleting a nonexistent field currently returns NA, where it should return the original object.  As an example:
DeleteField({"a":1, "b":2}, .c)

Spark returns Data.NA from the above.  The result should be {"a":1, "b":2}

Fixing this enables orderedWildcardWithProjection.test and sortWildcardOnExpression.test on next-major.
QScript is deleting a nonexistent field (which is incorrect, though not strictly so), which was hitting this spark bug. This QScript issue will be tracked separately.

The QScript issue is tracked in #2703.


Should provide command flags for redis 6

I've noticed that the list of flags returned by COMMAND for cl.throttle in redis 6.x is empty. There is a new CreateCommand function for Redis 6 which takes a list of command flags that apply to the command. I think @write and @fast apply.
https://redis.io/topics/modules-api-ref#coderedismodulecreatecommandcode
This may be related to issues I'm having with redis-cell and lettuce on redis 6, but I'm not sure: https://github.com/lettuce-io/lettuce-core/issues/1327

Looking at the code, the flag list is provided with write, which makes me wonder if there is a bug instead?
 72) 1) "cl.throttle"
     2) (integer) -1
     3) 1) write
     4) (integer) 0
     5) (integer) 0
     6) (integer) 0
     7) (empty array)

Interesting — I hadn't taken much of a look at Redis 6 yet, and this is the first time I've seen these flags.
There's no documentation about them in COMMAND, but they do appear in the example run near the bottom of the page. They do get a brief mention in code that suggests they're used for ACL categorization.
I'm not sure there's much that can be done for redis-cell though. The exported CreateCommand API from Redis for Modules still only takes the original arguments, and I couldn't find a new version of it. I suspect that this is a relatively new feature that is not supported for Redis Modules yet, and not even well-described in the core yet.

Yes, I noticed due to the linked bug in lettuce handling empty arrays, but wasn't able to find more information. I originally confused them with the flags under 3 when I opened the ticket; and backtracked when I found you were already providing those and it didn't help.
There's some mention of a new module api in an early announcement about 6 but I couldn't find anything about that either. I'll ask around.


levelx rewrite a nand physical page twice！！！

when i tested the sample nand simulator，i got an error：
_lx_nand_flash_simulator_system_error block:0 page:0 error_code:5
when format media，_lx_nand_flash_open is called，it writes the initial erase count 1 into  block n，page 0.
when the block is full,  it rewrites the page 0, the error occurs.  Also, i found that levelx rewrites the nand spare several times.
As i know, one nand page is writted, we should erase the block before write the same page again.

@yinglli thank you for the feedback, we are working on single write implementation on nand flash

This issue is fixed in release v6.2.1


can't export and import to itemtemplate?

hi.
I use this plugin in VS2015.
But, I couldn't export and import to itemtemplate in nodejs project.
Are these behavior normal?

Indeed, you are correct... my best guess is that we identify and see templates for Node.js, not JavaScript, so we'd have to investigate adding this feature.
@SteveALee - mea culpa, I may have misled you the other day when I linked you to some docs on creating project templates.

Yeah, I want to add this feature too.

Closing this for now. It involves much more work than expected, for relative little gain. More info on creating your own templates can be found here: How to: Create Item Templates


Migrate AB3 to LinkyTime diagrams in DG

Related issues

For #58.
Description

Convert the following in DG from AB3 to LinkyTime:

[x] UI component docs and diagram
[ ] Parser docs and diagram
[ ] Storage component docs and diagram
[ ] Sequence diagram

Codecov Report

Merging #208 (349b534) into master (ea8bf7a) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #208   +/-   ##
=========================================
  Coverage     71.61%   71.61%
  Complexity      560      560
=========================================
  Files            94       94
  Lines          1878     1878
  Branches        202      202
=========================================
  Hits           1345     1345
  Misses          447      447
  Partials         86       86

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ea8bf7a...349b534. Read the comment docs.