AmazonMQ broker not sync due to permission problem ec2:ModifyNetworkInterfaceAttribute

Describe the bug
Trying to create Amazon MQ for new EKS workshop using ACK, and resources doesn't sync correctly.
Trying to create broker on the same VPC as the EKS cluster, using private subnet, and creating security group.
Steps to reproduce
Create Security Group
apiVersion: ec2.services.k8s.aws/v1alpha1
kind: SecurityGroup
metadata:
  name: "mq-eks-workshop"
  namespace: default
spec:
  description: SecurityGroup mq-eks-workshop
  name: mq-eks-workshop
  vpcID: vpc-07488ff69b233a7c0
  ingressRules:
  - ipProtocol: tcp
    ipRanges:
    - cidrIP: "<IP_ADDRESS>/0"
    fromPort: 61616
    toPort: 61619
  - ipProtocol: tcp
    ipRanges:
    - cidrIP: "<IP_ADDRESS>/0"
    fromPort: 8162
    toPort: 8162

Create Broker
apiVersion: mq.services.k8s.aws/v1alpha1
kind: Broker
metadata:
  name: "mq-eks-workshop"
spec:
  name: "mq-eks-workshop"
  deploymentMode: SINGLE_INSTANCE
  engineType: ActiveMQ
  engineVersion: "5.15.8"
  hostInstanceType: "mq.t3.micro"
  publiclyAccessible: false
  autoMinorVersionUpgrade: false
  users:
    - password:
        namespace: default
        name: "mq-eks-workshop"
        key: password
      groups: []
      consoleAccess: true
      username: admin
  subnetIDs:
  - subnet-091dc19cd206af16f
  securityGroups:
  - sg-020681d6cedde4a28

Expected outcome
Resource to be sync
Environment
Installed MQ controller, and setup iam with suggested iam role permissions https://github.com/aws-controllers-k8s/mq-controller/blob/main/config/iam/recommended-policy-arn
arn:aws:iam::aws:policy/AmazonMQFullAccess

Kubernetes version
EKS 1.23
AWS service targeted (S3, RDS, etc.)
MQ

Resources status
    conditions:
    - message: |-
        ForbiddenException: Access Denied: You aren't authorized to perform this operation. You must have permission to call ec2:ModifyNetworkInterfaceAttribute.
        {
          RespMetadata: {
            StatusCode: 403,
            RequestID: "83aa8347-c132-429d-aa2e-35a12d086ff1"
          },
          Message_: "Access Denied: You aren't authorized to perform this operation. You must have permission to call ec2:ModifyNetworkInterfaceAttribute."
        }
      status: "True"
      type: ACK.Recoverable
    - lastTransitionTime: "2022-11-01T15:37:22Z"
      message: Unable to determine if desired resource state matches latest observed
        state
      reason: |-
        ForbiddenException: Access Denied: You aren't authorized to perform this operation. You must have permission to call ec2:ModifyNetworkInterfaceAttribute.
        {
          RespMetadata: {
            StatusCode: 403,
            RequestID: "83aa8347-c132-429d-aa2e-35a12d086ff1"
          },
          Message_: "Access Denied: You aren't authorized to perform this operation. You must have permission to call ec2:ModifyNetworkInterfaceAttribute."
        }
      status: Unknown
      type: ACK.ResourceSynced

We have to update the iam recommended policy to be a recommended-inline-policy like we have here https://github.com/aws-controllers-k8s/lambda-controller/blob/main/config/iam/recommended-inline-policy


Help with HomeAssistant

Hello,
I know this isn't directly related to your library, but I'm hoping you can give me hints anyway. I've tried all of the August/Yale integrations on HomeAssistant, and can't log into any of them successfully. However I pulled down your library locally and was able to connect to my Yale locks. I'm new to HomeAssitant, so is your library already available as an integration? If not, how do I add it?

I'm not sure what happened, but the "August" integration in HomoeAssistant is now working. Sorry about that.


Explorer wallet connection is not properly styled

Describe the bug

To Reproduce
Steps to reproduce the behavior:

Go to explorer.testnet-8.union.build
Click on connect wallet

Expected behavior
legibility
Screenshots
If applicable, add screenshots to help explain your problem.
Device (please complete the following information):

Hardware: MacBook Pro
OS: macOS
Browser: Arc

https://github.com/unionlabs/explorer/pull/7


How can I make him not so big when there are only a few candles? Looking forward to your reply

use xEvents to resolve this

use xEvents to resolve this


chore(deps): upgrade dev dependencies

Upgrades project dependencies. See details in workflow run.

Automatically created by projen via the "upgrade-dev-dependencies-1.x" workflow

⚪ Backport skipped
The pull request was not backported as there were no branches to backport to. If this is a mistake, please apply the desired version labels or run the backport tool manually.
Manual backport
To create the backport manually run:
backport --pr 1540

Questions ?
Please refer to the Backport tool documentation


question: jwt integration

The login Caddy plugin creates jwt_token cookie. The jwt Caddy plugin reads the JWT token in the cookie and authorizes access.
The login plugin creates JWT token using HMAC algo (HS512). In order to compute an HMAC both plugins need a secret key.
What is the expected interaction between login and jwt plugin? How do the two make sure that the secret key is in sync?
cc: @g-w

Stumbled upon the following comment by @roblabla. It answers my question:
caddy-jwt and loginsrv have a cool "feature" where loginsrv will automatically generate a random jwt
secret if none are set in the Caddyfile, and store it in the JWT_SECRET environment variable. When
caddy-jwt later parses its own directive, it will use the secret found in the JWT_SECRET environment
variable if none are configured.

Now, this is obviously ordering-dependent. The login directive must be parsed before the jwt
directive for this to work. Which, obviously, doesn't really work out reliably well. I'm not sure
how caddy parses the directive and how everything is ordered, but when I tried it,
caddy-jwt didn't find any JWT_SECRET because it was parsed before loginsrv

FWIW the second paragraph refers specifically to my attempt at making that feature work in my Caddy2 fork of the two modules. It works with 100% reliability in the Caddy1 version found in this repo.


Programmatically add network

Hey, do we have API to add new network e.g. subnet programmatically? I'd like to build similar app like https://chainlist.org/ but for Stacks-based network.

Thanks for the request. We're tracking this enhancement here: https://github.com/hirosystems/wallet/issues/2535
If you're interested in working on it, we'd love to review a PR that attempts to add such an API!

Hi @markmhx, I think that's an interesting issue to pick up. I'm not sure if I have bandwidth to do it, but will open a PR if I have.

I'm closing in favor of https://github.com/leather-io/issues/issues/279#issuecomment-2357804473


Add more Ubuntu versions and Debian support

Individual commits detail each change.
Basically:

Added some best-effort support to support older go verisons that may not support the GOMODCACHE env var
Sets $PATH in the deb build scripts to what's in the worker image to improve reproducibility
Add support for non-standard Ubuntu/Debian go packages from backports that do not install go into $PATH
Add Ubuntu versions
Add Debian versions

Updated with suggestions.


add Global genes as user on website

https://globalgenes.org/

What should the description be?
From their website:
A non-profit organization dedicated to eliminating the burdens and challenges of rare diseases for patients and families globally.
@nicolevasilevsky does it sound good?

Perfect! I usually use the descriptions from the website for each group.


Axiom Equality Constraints

Given requirements for interoperating between axiom gate chips and standard halo2 chips (given with example of keccak rather than StandardPlonk, but same process):

Do halo2-lib computations, make calls to KeccakChip to have keccak do witness generation, return all witness cells to be assigned in keccak part of the arithmetization
Actually assign keccak part in vanilla halo2, returning AssignedCells: https://github.com/axiom-crypto/axiom-eth/blob/6d2a4acf559a8716b867a715f3acfab745fbad3f/src/keccak/mod.rs#LL751C35-L751C35
Need to do some tricks to convert AssignedCells to AssignedValues (basically just some way to make copy constraints between halo2-lib and keccak cells): https://github.com/axiom-crypto/axiom-eth/blob/6d2a4acf559a8716b867a715f3acfab745fbad3f/src/keccak/mod.rs#L423
Now assign halo2-lib AssignedValues into vanilla halo2, where it already has access to the assigned keccak cells, so it can do copy constraints appropriately: https://github.com/axiom-crypto/axiom-eth/blob/6d2a4acf559a8716b867a715f3acfab745fbad3f/src/keccak/mod.rs#LL778C18-L778C18

Likely requires a custom thread builder!

Hey @jp4g ! Saw your comment on this in the halo2-lib discussion TG as well. Would you require help with this? Would be delighted to contribute

Hey @jp4g ! Saw your comment on this in the halo2-lib discussion TG as well. Would you require help with this? Would be delighted to contribute

DM'd on telegram, ty! If you'd prefer to do things publicly in this issue though we can work from this issue


An uninstall dry-run flag when uninstalling shared dependencies

When uninstalling shared dependencies (#166) it would be useful to have a dry-run option that would output the shared dependencies and ordering they would be uninstalled. This should be in both Go API and CLI command form (list, YAML, and JSON output).

my two cents here:
The hypper uninstall can have a confirmation step that lists out the resources to be deleted and prompts for yes/no. And of course, a --no-prompt or --force flag for skipping the confirmation.
(I have not looked at the code closely so please ignore if this is what hypper already has)


Add initial upgrade testing for Data Science Pipelines

Adds initial upgrade testing for Data Science Pipelines, creating a pipeline server and starting a long running pipeline, and checking the status after upgrade.

Note: Aditional scenarios, like setting up an scheduled run before upgrade, will be covered in follow-up PRs

Use Verify Run Status in DataSciencePipelinesBackend
Set pip_trusted_host in gpu availability sample pipeline

This PR (before the latest rebase) was tested successfully in rhoai-test-flow/617


Generate changelog diff

Have to think out the flow a little bit more, but as a rough idea:

user runs dfetch check
notices a dependency was updated
runs dfetch check --with-changelist
for each project with a changes or changelist document at top-level show the textual diff between the current version and the available version.

Also see https://github.com/dependabot/dependabot-core/tree/main/common%2Flib%2Fdependabot%2Fmetadata_finders%2Fbase for an approach


boost parsing performance with parallel parsing support

This is a big change to general approach of parsing excel files.
It extracts zip into memory, writes directly to ets tables avoiding sending messages to agents. On application start it creates a single state worker which is a parent for all ets tables..
According to my benchmark on the same ~12 Mb excel file:
was: 2:40m
now: 22s

#49

Hey @AlexKovalevych thanks for all the work on this. My apologies for the delay in replying, I've been trying to wrap up my busy season at work. I plan to pull down and review your pull request soon. I'll let you know if any questions come up.

@AlexKovalevych there are a few issues we need to look into before I can merge this pull request:

While your changes greatly increased performance during the actual parsing of the file, it decreased the performance of the get_list() function. This is, of course, to be expected for the most part since you moved the generation of the nil cells to get_list(), there are situations where it becomes unusable. For example, in the test file (./test/test_data/test.xlsx) the worksheet at index 2 seems to get hung up when calling get_list() after extraction when it only took a couple of seconds before.

There are situations where no styles or shared_strings are generated due to the formatting of the file. These situations currently throw an argument error.

Since a generic :worksheet ETS table is no longer generated when using extract/3, there doesn't seem to be a need for it anymore. Its purpose was to negate the need for using and keeping track of a pid when working in a single worksheet.

Documentation and testing needs to be updated to incorporate your changes.

I'll keep testing to see if I can come up with anything else. Thanks again for all the work you put into this!

@jsonkennell Thank you for your review. I'll work on the issues you found.

@jsonkennell I've fixed filling cells with nils (it was infinite recursion in some cases), but could you give me an excel file example without styles or shared_strings (or maybe create such test) so i can reproduce the issue.

@AlexKovalevych I just created a branch called no-shared-strings where I added an .xlsx file for you to reproduce the issue. ./test/test_data/no-shared-strings.xlsx

@jsonkennell i fixed the issues and updated documentation a bit, please let me know what else should be done to complete this PR (I didn't update the HTML documentation though, should I? )

@AlexKovalevych no, don't worry about the HTML docs, I'll update that before I publish to hex. I have a couple other ideas I'd like to work on with this PR before we merge. I'll try to work through them and get you comments over the upcoming weekend.


SubmissionError with sync submit handler

asked this on SO but didn't get a response there: http://stackoverflow.com/questions/39164925/submissionerror-with-sync-submit-handler-in-redux-form

On redux-form, I can propagate a general form error from an async submit handler by throwing a SubmissionError just like described here. However, this doesn't seem to work when my submit handler doesn't return a promise. Is there way to achieve the same error propagating behavior of async submit handler with sync submit handler?

Your onSubmit function:
        return new Promise((resolve, _reject) => {
                throw new SubmissionError({
                    fieldname: 'some error',
                    _error: 'Form could not be submitted.',
                });
        });

Basically create a new promise that is returned, and inside that promise throw the SubmissionError. Not the most elegant, but it works.

@bertjwregeer seems like a good enough workaround, thx! I was gonna go with that but wanted to see if there is a legitimate way to do this.


Proposal: Remove SA1402 exclusion of "helper types"

Currently SA1402 is written such that it only applies to classes. However, SA1649 applies to the first type in a file. As such, there is a fundamental conflict between rules SA1201, SA1402, and SA1649.
I propose modifying SA1402 to no longer allow a file to contain more than one top-level type, regardless of the kind of type. The primary rationale is that a type which is directly related to the top-level type such that it shouldn't be removed from the file should actually be a nested type, which is already supported without problems by the rules mentioned above.

:+1: Agree. I would consider exempting delegates, as they are not on the same level as classes, interfaces, structs and enums. See also my comment in #1757.

We'd probably write our own variant of this rule to avoid the file explosion it'd cause in our projects. We've already done so with SA1649 partly to avoid this conflict.

I think the exception for delegates would be common, so I'm narrowing this proposal to simply considering interface, struct, and enum types (in addition to class which is already supported).

interface, struct, and enum should all be in their own files. :+1:

I can fix this.
KnownChanges does not mention SA1402 right now, so I guess this is a change away from Classic's behaviour? Change the class name of the analyzer as well, to match its new behaviour?

We could also make this configurable:
"topLevelTypes": [
  "class",
  "interface",
  "struct",
  "enum",
  "delegates"
]

Where the default is [ "class" ] (preserves the current behavior).
@appel1 Would this be sufficient to cover your special cases?

@sharwell Yes, that would work for us.

That looks like the way to go.

Assigning to @bjornhellander. Let me know if you have any questions! :smile:

I would expect both SA1402 and SA1649 to be affected by the proposed configuration. Do you agree?


Ability to set separate themes per cluster

What kind of request is this (question/bug/enhancement/feature request):
Feature Request
I'd like to have the ability to set different themes per cluster.
For example, I would like to be able to make my production clusters very visually distinct from my development clusters.

See https://github.com/rancher/rancher/issues/15838


Add the ability to ignore a given line of code with a comment

Just as eslint has a comment directive to disable a given line, this
commit adds a i18n-extract-disable-line comment to disable extraction
froma given line of code.
This fixes #32.

This PR looks great :D.

Thanks!

Do you want me to release it or do you want to tackle the other issue?

I think I can handle the other issue, see https://github.com/oliviertassinari/i18n-extract/issues/31#issuecomment-320046332. It would be better to have a release with both issues solved :)


Account for env var RUSTC_BOOTSTRAP when determining if features are supported

Fix https://github.com/SergioBenitez/Pear/issues/18.

Is there official documentation for the RUSTC_BOOTSTRAP environment variable anywhere? I'd like to know when it was introduced, for what reason, and whether it is something that will remain in the compiler. In other words, I'd like to better understand the purpose of the flag.

RUSTC_BOOTSTRAP=1 is used to be able to build any version of rustc using a compatible stable rustc. It tells a stable build of rustc to behave as if it were a nightly build, allowing unstable features to be used. It should't be encouraged, but it is a tool that is available. Supporting this makes version_check behave inline with rustc. Given that supports_features is a convenience to verify whether unstable features will be able to be used in the current compilation, I am inclined to argue that this new behavior is the correct one.
https://github.com/rust-lang/rust/issues/31847#issuecomment-280666952 hints at the unlikelihood of this var going away in the short term.
The (lengthy) discussion in https://github.com/rust-lang/cargo/pull/6608 and https://github.com/rust-lang/cargo/issues/6627 elaborates on the current uses and counter arguments against using this environment variable.
The "documentation" of RUSTC_BOOTSTRAP is only the code (😢):
https://github.com/rust-lang/rust/blob/885c7dfddcca11fa543d99d702c024c3f6bfe07b/src/libtest/lib.rs#L515-L523
https://github.com/rust-lang/rust/blob/8e948df707ea8a3c88c65bf2ffdcb2f1cf5491be/src/libsyntax/feature_gate.rs#L2534-L2553
It also seems to me that support for CFG_DISABLE_UNSTABLE_FEATURES=1 should be added too to fully map against rustc's behavior.

@estebank Is RUSTC_BOOTSTRAP intended to be used by end-users?
In any case, the use in the code snippets you refer to seems to imply that setting RUSTC_BOOTSTRAP effectively makes the stable compiler act like a nightly compiler. As such, I would argue that the correct way to support this feature is to modify Channel::read() accordingly, making a note of the behavior.

@estebank Is RUSTC_BOOTSTRAP intended to be used by end-users?

It is not necessarily, but it is being used (most notably by Firefox), but it immediately lowers the assurances that stable rustc gives to those of nightly. This is used in environments where people wishes they could be using stable, but are reliant on a subset of unstable features. https://github.com/rust-lang/cargo/issues/6627 is an attempt at formalizing this in a controlled manner, and the pushback is against making RUSTC_BOOTSTRAP part of our stable API.

In any case, the use in the code snippets you refer to seems to imply that setting RUSTC_BOOTSTRAP effectively makes the stable compiler act like a nightly compiler. As such, I would argue that the correct way to support this feature is to modify Channel::read() accordingly, making a note of the behavior.

That sounds reasonable to me, but I'm not sure. I believe that the primary intent for the flag is to affect features.

@estebank Is RUSTC_BOOTSTRAP intended to be used by end-users?

It should unequivocally not be used by end users.
The flag may be renamed, removed, checked for tampering, etc. at any time.
I'm against giving RUSTC_BOOTSTRAP any more visibility than it already has.
I think it should remain something obscure.

Apologies. I meant to close this nearly a year ago. I don't think we should be listening to hidden/private values that may change, especially since this can be done on a case-by-case basis.


The file_input.vue has bee removed from the latest build

In order to work the file_input widget require a file_input.vue in the /usr/local/lib/python3.6/dist-packages/ipyvuetify/extra.
Since the last build this file is missing.
The error comes from the ipyvuetify lib, the file is not loaded during thre pip installation (issue #124).
Can we manually downgrade the ipyvuetify installation to 1.5.1 until Mario find a solution ?

Will be included in next release.

The problem have been solved in Ipyvuetify, In the next build you'll be able to let the automatic update to 1.6.2 (will be available in couple of days)


Storage throws an NPE on getting the broker offsets

The storage module can throw an NPE while fetching a consumer's information if the consumer has committed an offset for a partition that does not yet have a broker offset. This is due to an assumption in fetching the offset ring that the ring has been created.
By making sure we create the offset ring earlier we can make that assumption valid.

Coverage increased (+0.02%) to 73.963% when pulling 852c0b4084ce174d16284b0e2323eb26765d4854 on toddpalino:inmemory-brower-ring-npe into 61f097a4203ea4001b4214c9ceb25d8b5ac6ee20 on linkedin:master.


Fix for snap-1507

Changes proposed in this pull request
Fix for snap-1507.  Route the query to Spark engine if an error is thrown as GemXD does not support more than one level of nested subquery.
Tests can be referred in https://github.com/SnappyDataInc/snappydata/pull/583
Patch testing
Added a unit test
snappy precheckin
ReleaseNotes changes
None
Other PRs
https://github.com/SnappyDataInc/snappydata/pull/583

good to go


DOC: Add links to colab in notebook

Adds links to colab and github in the beginning of notebooks.

Thanks!


Tweaks after install to make Mopidy work

Hi all!
First things first: Great project! Thanks so much!
Problem: Due to a broken SD card I had to reinstall my box. Now I am struggling with the new version and I just can't get it to play any files. If I use the shell to add songs to mpc and play them it works. But something seems broken when playout_controls.sh tries to control mpc...
Nevertheless I can play streams! But no .mp3s.
I switched from devel to master but it made no difference.
I would be thankful for any debugging hints. I have in depth Linux skills. I already set debug for the script but it does not show anything helpful.
Version: 2.0 rc7
Edition: Plus edition (feat.
If I use mpc load FooBarand then mpc playlist I will see nothing.
Why can't mpc read its own playlists?!
pi@phoniebox:~/RPi-Jukebox-RFID/playlists $ cat FooBar.m3u
local:track:FooBar/startupsound.mp3

After I changed the playlist_recursive_by_folder.php and use the "classic" way and not the "plusSpotify" way to generate the playlists, MPD works as expected (?).

I think I found the root of the problem:
Since Mopidy 3.x was released, they dropped Python 2.7 support and some required plugins (IRIS and the Playlist parser) are not yet compatible with Python 3.7.
This causes the strange behavior I discovered (only webstreams work, no local files).
You can check if you stop the mopidy daemon and start mopidy on the command line. For me it then prints a bunch of errors when importing the plugins.

same issue here

does it work for you after downgrading mopidy to the latest 2.3 version?
sudo apt-get install mopidy=2.3.1-1

@ZyanKLee Thanks for the hint, already tried that. Fails with tornado:
pi@phoniebox:~/mopidy-spotify $ mopidy
INFO     Starting Mopidy 2.3.1
ERROR    Failed to load extension iris: invalid syntax (web.py, line 205)
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mopidy/ext.py", line 202, in load_extensions
    extension_class = entry_point.resolve()
  File "/usr/lib/python2.7/dist-packages/pkg_resources/__init__.py", line 2417, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python2.7/dist-packages/mopidy_iris/__init__.py", line 5, in <module>
    import tornado.web
  File "/usr/local/lib/python2.7/dist-packages/tornado-6.0.3-py2.7-linux-armv7l.egg/tornado/web.py", line 205
    application: "Application",
               ^
SyntaxError: invalid syntax
INFO     Loading config from builtin defaults
INFO     Loading config from /home/pi/.config/mopidy/mopidy.conf
INFO     Loading config from command line options
ERROR    Validating extension http failed with an exception.
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/mopidy/ext.py", line 272, in validate_extension_data
    data.extension.validate_environment()
  File "/usr/lib/python2.7/dist-packages/mopidy/http/__init__.py", line 34, in validate_environment
    import tornado.web  # noqa
  File "/usr/local/lib/python2.7/dist-packages/tornado-6.0.3-py2.7-linux-armv7l.egg/tornado/web.py", line 205
    application: "Application",
               ^
SyntaxError: invalid syntax

No clue, what python does not like about that line...

My mistake: Iris requires tornado <= 5.0
After installing pip install 'tornado==5.0' the error is gone.

Thank you all for posting problem and solution, including what to type into the shell! This + https://github.com/MiczFlor/RPi-Jukebox-RFID/issues/700 made it work for me on RaspberryPi 2 and Raspbian Buster.
sudo apt-get install mopidy=2.3.1-1
sudo python2.7 -m pip install Mopidy==2.3.*
pip install 'tornado==5.0'

@FloMrFrei so you've ran it prior the oneline Installer?

You can even do so afterwards. At least I did and it worked.
Though I ran also the third command with sudo.

@JoeyRamone I run those lines after the 1-line installer. I guess otherwise, the installer will install mopidy 3.x over the requried 2.3.1 version.

If you want to prevent apt from upgrading Mopidy you can use apt-mark hold mopidy after installing your specific version.

The PRs #789 and #793 have been merged to master and develop.
Could you please check, if the issue is fixed now?

Closing as the info is very outdated.


Subtract DC level from light curves?

At the August 25th Twinkles weekly meeting, we identified a function we need the Monitor to perform: according to @rbiswas4 it is standard practice to subtract off the flux due to the host galaxy from SN forced photometry light curves. To implement this, we could have the Monitor find the "DC level" of a light curve (far from the SN brightening and fading) and subtract it from the light curve (propagating errors appropriately). We could avoid this if sncosmo perfomed fits that included a constant background level - does it?
As the seeing varies from visit to visit, the amount of galaxy flux brought into the flux measurements will change - but perhaps this can be taken to be an additional noise term? Advice would be most welcome before we code this, @rbiswas4 @wmwv

To implement this, we could have the Monitor find the "DC level" of a light curve (far from the SN brightening and fading) and subtract it from the light curve (propagating errors appropriately). We could avoid this if sncosmo perfomed fits that included a constant background level - does it?

It does not in the standard implementation
I was suggesting something different actually (simpler or more complicated).

Do forced photometry at the location at all other times when the SN is not present, and average it to call the flux contribution of the galaxy, with errors.
Subtract  this number from what we are calling the light curve fluxes, as you would if you subtracted two estimators. This will lead to both smaller fluxes for the SN, and higher error bars.
Do usual light curve fits

This is simpler because you directly estimate the galaxy flux without inferring it from measuring SN + galaxy flux and a model. It is more complicated in that you have to run the DM stack and find times where there are no SN (easy in reality, hard in that run).

As the seeing varies from visit to visit, the amount of galaxy flux brought into the flux measurements will change

Exactly. The flux contribution from the host depends somewhat on the observing conditions. This is why people want to try to model this using scene modeling for things like a final analysis. But this is completely lost in either the correction I suggested, or in the sncosmo fit method you proposed because it also assumes a constant background level.
As I was writing, I also think we can estimate the galaxy contribution (in a way not accessible to observers) by using the galaxy surface brightness and PSF in the relevant visit to calculate the galaxy contribution. (Is this already done in DM as model flux?) It would be a useful thing to have.
Also, I do think that the idea of implementing a fit for SN parameters and zero points simultaneously is useful and not too difficult (I even think this was done in some hack session but not taken to completion). Since you are in a astrohackweek next week, maybe this could be a project ? :)

Image subtraction or scene modeling are the only ways to do this right.  I don't think it's worth spending much effort trying to do this "right" by doing forced photometry on unsubtracted images.  In particular if the goal is to do light-curve fits to recover parameters to try to learn how we're doing, I don't think we can learn that much quantitatively useful until we do either scene modeling or image subtraction.

OK: that says that maybe a simple DC level correction like the one Rahul
suggested is all that is needed to make our current ForcedSource light
curves correspond to the current industry standard - or would you say that
the current industry standard is different?
I am all for pursuing image differencing, but we got talking in the meeting
about fallback options, and what DM Level 2 analysis to do. If image
differencing is not enabled until November, there could be something to be
said for trying to do our current DM pipeline processing before then, to
get it set up. What do you think?
On Thu, Aug 25, 2016 at 6:48 PM, Michael Wood-Vasey <
<EMAIL_ADDRESS>wrote:

Image subtraction or scene modeling are the only ways to do this right. I
don't think it's worth spending much effort trying to do this "right" by
doing forced photometry on unsubtracted images. In particular if the goal
is to do light-curve fits to recover parameters to try to learn how we're
doing, I don't think we can learn that much quantitatively useful until we
do either scene modeling or image subtraction.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/DarkEnergyScienceCollaboration/Monitor/issues/50#issuecomment-242600040,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AArY9_fBlFEoroWmWAAZyUoMDovJ40l5ks5qjkXmgaJpZM4JtpYz
.

AFAIK the industry standard for publishing SN photometry is doing SMP (SDSS/SNLS/DES) went that route .... so I agree with @wmwv  that doing forced photometry and galaxy subtractions will not be doing it 'right'. At the same time, forced photometry based results have been used before such a finalized version just to get analyses going. I would be very interested in working on a problem like SMP, but I don't know what the word on that is, and one where access to real data at high z might be important.
At the end of the day, we will be using a method which is more 'right' than what I described. So there is a sense in which that work would go to waste. So, it might be better to have me (since I was assigned this) try to push on diff im which will be useful to get going on Twinkles (for example it would be useful to do detections using difference images and comparing with out cheat mode).


 Given: 4194

Expected behavior
I understand the length of the test prompt exceeds max-input-length and max-total-tokens, but why the validations for these two commands are different?

I don't have access to both repos, but most likely there' s a difference in the tokenizers.
For instance the first one might NOT have a fast tokenizer (tokenizer.json), which prevents the router from actually counting tokens (meaning it will assume max tokens for the query and simply pass it along)

After checking both repos, the first one does NOT have fast tokenizer (tokenizer.json) while the second one does have it. so as your guess, this might be the reason why the first one does not throw any error.
If this is the case, I'm a bit confused about the current way to activate the rope scaling.
In this TGI's official documentation, it suggests making sure to pass --max-input-length to provide model’s original max sequence length for extension. I set --max-input-length as 2048 because it is the "max_position_embeddings": 2048 defined in the model's config.json. By setting --rope-scaling dynamic and --rope-factor 2.0 it should be able to handle longer prompts (> 2048 tokens). However, it looks like the current router's validation will turn down those long prompts and throw the input validation error (assume that the used model has a fast tokenizer). Look forward to your verification on this.
Thank you.

Make sure to pass --max-input-length to provide maximum input length for extension.

This should be understood as pass in 4096, no ?

@Narsil Thank you for your clarification. I think I misunderstood the document at first.
Now I'm clear that --max-input-length is for router's validation. I found in your rope scaling implementation, TGI will read config.max_position_embeddings for max_position_embeddings. At first I thought --max-input-length is relevant to rope scaling, but just realize it is not.


fix(runtime-dom): fix getContainerType

TL;DR async components with lazy hydration break when interacted with very quickly after hydration
The getContainerType function does not accept null, however in packages/runtime-core/src/hydration.ts:281 the following code const container = parentNode(node)! assumes container will not be null, but in fact, it is, and reading a field from it in getContainerType fails. I do not understand the need to use the non-null assertion operator throughout the code base of Vue, but I guess sometimes it can be OK for performance reasons.
Something else I do not understand is, after fixing getContainerType locally, we did not experience any further crashes (for now), and looking at the code, mountComponent expects to always get a container, but whenever that happens (container = null), and it does happen now and then since getContainerType used to crash in 1 of 5 test runs, no observable malfunctions happen. How can mountComponent still function properly when it requires a container, but gets passed a null?!
TypeError: Cannot read properties of null (reading 'nodeType')
    at getContainerType (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:3781:17)
    at hydrateNode (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:3943:13)
    at hydrateSubTree (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:7440:13) unknown {"name":"TypeError","message":"Cannot read properties of null (reading 'nodeType')","trace":"TypeError: Cannot read properties of null (reading 'nodeType')\n    at getContainerType (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:3781:17)\n    at hydrateNode (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:3943:13)\n    at hydrateSubTree (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:7440:13)"} Error: Uncaught TypeError: Cannot read properties of null (reading 'nodeType') @ File https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:3781:17 https://acme.local:3000/wedding-rings/mens/platinum
Cannot read properties of null (reading 'nodeType')
TypeError: Cannot read properties of null (reading 'nodeType')
    at getContainerType (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:3781:17)
    at hydrateNode (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:3943:13)
    at hydrateSubTree (https://acme.local:3000/node_modules/.vite/deps/chunk-TBLKTZOQ.js?v=11ed4a1a:7440:13)

# 3780 through 3785
var getContainerType = (container) => {
  if (container.nodeType !== 1) return void 0;
  if (isSVGContainer(container)) return "svg";
  if (isMathMLContainer(container)) return "mathml";
  return void 0;
};

# 3927 through 3956
        } else if (shapeFlag & 6) {
          vnode.slotScopeIds = slotScopeIds;
          const container = parentNode(node);
          if (isFragmentStart) {
            nextNode = locateClosingAnchor(node);
          } else if (isComment(node) && node.data === "teleport start") {
            nextNode = locateClosingAnchor(node, node.data, "teleport end");
          } else {
            nextNode = nextSibling(node);
          }
          mountComponent(
            vnode,
            container,
            null,
            parentComponent,
            parentSuspense,
            getContainerType(container),
            optimized
          );
          if (isAsyncWrapper(vnode)) {
            let subTree;
            if (isFragmentStart) {
              subTree = createVNode(Fragment);
              subTree.anchor = nextNode ? nextNode.previousSibling : container.lastChild;
            } else {
              subTree = node.nodeType === 3 ? createTextVNode("") : createVNode("div");
            }
            subTree.el = node;
            vnode.component.subTree = subTree;
          }

# 7413 through 7450
  const setupRenderEffect = (instance, initialVNode, container, anchor, parentSuspense, namespace, optimized) => {
    const componentUpdateFn = () => {
      if (!instance.isMounted) {
        let vnodeHook;
        const { el, props } = initialVNode;
        const { bm, m, parent, root, type } = instance;
        const isAsyncWrapperVNode = isAsyncWrapper(initialVNode);
        toggleRecurse(instance, false);
        if (bm) {
          invokeArrayFns(bm);
        }
        if (!isAsyncWrapperVNode && (vnodeHook = props && props.onVnodeBeforeMount)) {
          invokeVNodeHook(vnodeHook, parent, initialVNode);
        }
        toggleRecurse(instance, true);
        if (el && hydrateNode) {
          const hydrateSubTree = () => {
            if (true) {
              startMeasure(instance, `render`);
            }
            instance.subTree = renderComponentRoot(instance);
            if (true) {
              endMeasure(instance, `render`);
            }
            if (true) {
              startMeasure(instance, `hydrate`);
            }
            hydrateNode(
              el,
              instance.subTree,
              instance,
              parentSuspense,
              null
            );
            if (true) {
              endMeasure(instance, `hydrate`);
            }
          };

Please provide a minimal reproducible demo. This will help us troubleshoot and identify the root cause of the issue. Error logs alone make it difficult for us to pinpoint the exact problem.

Generally speaking, the exclamation marks are the main problem.
I will try to do so as soon as possible, but the minimal reproduction of this one is tough because of app routing and a myriad of things that need to be ported in it, that could be in a way or another leading to this.
And, by the way, https://github.com/vuejs/core/issues/7086 this is still broken with latest Vue. The problem is exactly the same, just elsewhere. There's a reproduction repo for 2 years posted now, and noone seems to care enough. I need to know whether the half-working-day of porting our app's internal structure will go to waste or not.

@wucdbm
I think #7086 hasn't been resolved in two years likely because the reproduction demo is too complex. it contains many third-party libraries. This wastes a lot of time for contributors trying to help you find the bug. This is unreasonable because you are most familiar with your codebase, and if you can't extract a minimal reproduction demo, it's even more impossible for others to do so.
please see https://antfu.me/posts/why-reproductions-are-required

I can understand that reproductions are required, I am working in the same field in the end and face the same issues.
For https://github.com/vuejs/core/issues/7086 the reproduction demo there is quite simple. Third party libraries are qutie irrelevant. I will then get rid of GraphQL and make a simple setTimeout() data fetcher. The thing of substance there is that the server does NOT see the hash parameter passed and consequently the server renders one thing, while the client expects to render something else due to the fact that it does see the query param and the UI must be different.
For this one, it'll take me a considerable amount of time to copy/pasta the project structure in terms nested transition/suspense/router-view/transition/suspense/keep-alive/suspense etc, to make an accurate representation of the conditions where it does break. And then, it breaks every 5th run of a test case. The test hovers over an component that is async with lazy hydration very quickly. Every now and then, the above happens.


ggplot 2 cheatsheet hard to find

the cheat sheet for ggplot "data-visualization-2.1.pdf " doesn't contain ggplot in the title, which when searching for the package's name could be a bit confusing. rename cheat sheet file to 'ggplot2' or 'data-visualization-with-ggplot2-2.1.pdf'?

@tallguyjenks I see your point, but changing the filename on github would break existing links, and there is no way I can tell who linked to what. In the future, I'll use the package name as the cheatsheet name if the topic:package mapping is one to one (not always the case).


Mysql 5.7 requires order by clause in select DISTINCT statement (Paginator)

Hi,
There seems to be an issue with the Paginator. In mysql 5.7 it is required to have the order by fields to be present in the select statement when using DISTINCT. I have situated where the problem at:
https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/Tools/Pagination/LimitSubqueryOutputWalker.php#L276
By preserving the order by clause as described in issue #2630 it now fails in mysql 5.7. I tested by simply adding the ordered fields to the select statement and everything worked again.

This just means, that the mysql 5.7 s now working the same as other databases. IMO, the best fix would be to always preserve the order by.

Yes, the preserve should stay. Here is an example of what happens.
SELECT DISTINCT id FROM (subquery) dctrn_result ORDER BY last_name DESC;

This select fails in mysql 5.7 but when adding the last_name to the select statement it runs without any problems
Working example:
SELECT DISTINCT id, last_name FROM (subquery) dctrn_result ORDER BY last_name DESC;

Having the same issue after upgrading to Ubuntu 16.04. Ordering by anything other than the id, will produce this error, when using the paginator.

I changed the code to works for me.
What i changed?
https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/Tools/Pagination/LimitSubqueryOutputWalker.php#L377
I added this line
$sqlIdentifier += $this->refactorOrderForSelect($orderBy);
And i added this method
private function refactorOrderForSelect($orderBy)
{
    // Removing ASC or DESC
    foreach($orderBy as $order) {
        $order = preg_replace(array("/( ASC){0,1}/","/( DESC){0,1}/"), array("",""), $order);
        $orderSelect[] = $order;
    }
    return $orderSelect;
}

Doing this the fields in order by clause are added to select statement.
I know its a poor code. So i did not create a PR.
But, its a way to close this issue!

@diego-mazzaro I'd call that a hack rather than a solution to close this issue.
It may work for you but it doesn't fix the underlying issue(s) that may cause more problems in the future. Fixing problems with hacks will lead to more hacks and unmaintainable code, which nobody likes since ORMs can be a huge PITA anyway.

Yes @cH40z-Lord, i agree... I Just post this hack because I need my system works as it is not released a new version of Doctrine ...
If it is useless , forgiveness. I just tried to help.

I found a similar problem reported here and @jradwan's solution of adding
[mysqld]
sql-mode=""

to /etc/my.cnf fixed the
"ORDER BY clause is not in SELECT list, references column 'dctrn_result.title_1' which is not in SELECT list; this is incompatible with DISTINCT"
error for me.

On /etc/mysql/my.cnf, in the mysqld section, add sql-mode = ""
So it will look like
[mysqld]
...
sql-mode = ""
...

As this obviously is an issue with invalid generated SQL, I think a bugfix is needed? (see #5930 for an example of an invalid select statement)
Relaxing all SQL modes in MySQL so that even invalid SQL works without errors is no solution imho (not even temporarily).

Duplicate of #4846?

Thanks ! This worked for me :
[mysqld] ... sql-mode = "" ...
This is a hack, but will be sufficient waiting for the PR

https://github.com/api-platform/core/issues/965

Please be aware, that by setting [mysqld] sql-mode="" you're disabling safe-mode.
That means you're opening up to multitude of issues. To name a few - invalid dates, zero dates, NULL on division by zero. See the full list of available SQL modes for more information.
Ideally only disable the one offending feature as @apfelbox mentioned above.

@Devolicious could you update your first message with the warning message above or link to it? People seem to be blindly using the workaround without knowing the consequences :/ Not many people will scroll down here :(

Is there any intention to fix this issue? mySQL 5.7 has been out for > 2.5 years!
The (awful) solution I'm using can be found in this gist. I tried a number of permutations of SQL modes to try to keep some data-integrity guarantees but in the end gave up - be warned that if you use the above solution you've significantly weakened mySQL's already weak protections against garbage data.

@ptlis this is currently being fixed in https://github.com/doctrine/doctrine2/pull/6143

Handled in https://github.com/doctrine/doctrine2/pull/6143

This worked for me
$clonedQb = clone $qb;
$clonedQb->resetDQLPart('orderBy');
$paginator = new Paginator($clonedQb->getQuery());

Any example please to disable after postConnect in doctrine 2.5?
    /**
     * Triggers after connection to Database
     *
     * @param ConnectionEventArgs $args
     * @return \Doctrine\DBAL\Connection
     */
    public function postConnect(ConnectionEventArgs $args)
    {
        $connectionParams = $args->getConnection()->getParams();
        $connectionParams['driverOptions'] = [
            PDO::MYSQL_ATTR_INIT_COMMAND => "SET sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''))"
        ];
        $evm = new EventManager();
        return DriverManager::getConnection($connectionParams, null, $evm);

This still throws
Caused by
PDOException: SQLSTATE[HY000]: General error: 3065 Expression #1 of ORDER BY clause is not in SELECT list, references column 'dctrn_result.created_at_11' which is not in SELECT list; this is incompatible with DISTINCT

@fagacil just upgrade the ORM, really.

Any example please to disable after postConnect in doctrine 2.5?
    /**
     * Triggers after connection to Database
     *
     * @param ConnectionEventArgs $args
     * @return \Doctrine\DBAL\Connection
     */
    public function postConnect(ConnectionEventArgs $args)
    {
        $connectionParams = $args->getConnection()->getParams();
        $connectionParams['driverOptions'] = [
            PDO::MYSQL_ATTR_INIT_COMMAND => "SET sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''))"
        ];
        $evm = new EventManager();
        return DriverManager::getConnection($connectionParams, null, $evm);

This still throws
Caused by
PDOException: SQLSTATE[HY000]: General error: 3065 Expression #1 of ORDER BY clause is
 not in SELECT list, references column 'dctrn_result.created_at_11' which is not in SELECT list;
this is incompatible with DISTINCT

And this also does not helped
    /**
     * Triggers after connection to Database
     *
     * @param ConnectionEventArgs $args
     * @return \Doctrine\DBAL\Connection
     */
    public function postConnect(ConnectionEventArgs $args)
    {
        $connectionParams = $args->getConnection()->getParams();
        $connectionParams['driverOptions'] = [
            PDO::MYSQL_ATTR_INIT_COMMAND => "SET sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''))",
            PDO::MYSQL_ATTR_INIT_COMMAND => "SET session.sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''))",
            PDO::MYSQL_ATTR_INIT_COMMAND => "SET global.sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''))",
            PDO::MYSQL_ATTR_INIT_COMMAND => "SET GLOBAL sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''))",
            PDO::MYSQL_ATTR_INIT_COMMAND => "SET SESSION sql_mode=(SELECT REPLACE(@@sql_mode,'ONLY_FULL_GROUP_BY',''))",
        ];
        $driver = new Doctrine\DBAL\Driver\PDOMySql\Driver;
        return new Doctrine\DBAL\Connection($connectionParams, $driver);

and this...
mysql.conf
[mysqld]
user=mysql
default-storage-engine = INNODB
sql-mode = "STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"

Our query
 $queryBuilder = $this->entityManager
                ->createQueryBuilder()
                ->select('a')
                ->addOrderBy('a.createAt', 'ASC')
                ->from(User::class, 'a');
   return $this->paginate($queryBuilder->getQuery(), $pagination->limit(), $pagination->page());

Also tried with DQL,
 $queryBuilder = $this->createQueryBuilder()
                ->select('a')
                ->addOrderBy('a.createAt', 'ASC')
                 ->setLimit(5) // smth like that
                  ->setPage(1)
                ->from(User::class, 'a');
 return new Doctrine\ORM\Tools\Pagination\Paginator($queryBuilder, true);

Spent about 6 hours already. Using php 7.0, doctrine/dbal 2.5, docker image.
Anyone clear solution?
this not helped too, maybe coz docker
[mysqld]
...
sql-mode = ""
...

also trying to fix overhere
https://github.com/doctrine/doctrine2/blob/master/lib/Doctrine/ORM/Tools/Pagination/LimitSubqueryOutputWalker.php#L377
by this code

private function refactorOrderForSelect($orderBy)
{
    // Removing ASC or DESC
    foreach($orderBy as $order) {
        $order = preg_replace(array("/( ASC){0,1}/","/( DESC){0,1}/"), array("",""), $order);
        $orderSelect[] = $order;
    }
    return $orderSelect;
}

does not help
running this in docker mysql container does not helped
  mysql> set global sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
    mysql> set session sql_mode='STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION';
    mysql> exit;

same to
SET sql_mode = ''
removing line of addOrderBy helped, but we have to sort it in passed order, like some variable $fieldName and $sortDirection
Still got this cool error
Caused by
PDOException: SQLSTATE[HY000]: General error: 3065 Expression #1 of ORDER BY clause is not in SELECT list, references column 'dctrn_result.created_at_11' which is not in SELECT list; this is incompatible with DISTINCT

also tried to add in /etc/mysql/my.conf
.....
#
# * IMPORTANT: Additional settings that can override those from this file!
#   The files must end with '.cnf', otherwise they'll be ignored.
#

!includedir /etc/mysql/conf.d/
!includedir /etc/mysql/mysql.conf.d/

[mysqld]
sql-mode=""

not helped
Holly crap .
  database:
    image: mysql:5.7
    env_file:
      - ./stack/environment/development/database.env
    ports:
     - 8307:3306
    command: mysqld --sql_mode="STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"
    volumes:
     - data:/var/lib/mysql
     #- ./stack/mysql/config/my.cnf:/etc/mysql/my.cnf

Solved the issue, because container
To see what values a running MySQL server is using, type
'mysqladmin variables' instead of 'mysqld --verbose --help'.
root@594de3cb146d:/etc/mysql# mysqld --verbose --help

mysqld: [Warning] World-writable config file '/etc/mysql/my.cnf' is ignored. <--- NOTE THAT IGNORED

mysqld  Ver 5.7.21 for Linux on x86_64 (MySQL Community Server (GPL))
Copyright (c) 2000, 2018, Oracle and/or its affiliates. All rights reserved.

Because of that, container configs has
.....
sporadic-binlog-dump-fail                                    FALSE
sql-mode                                                     ONLY_FULL_GROUP_BY,STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
ssl                                                          TRUE
ssl-ca                                                       (No default value)
.....

Spent about 10 hours to fix one line of that . We do not know why it is ignored. If anyone has ideas, we would be glad to hear.

thank you very much. i got the same problem too when i use mysql 8 by docker-compose. it won't work if i set the sql_mode by edit my.cnf. it should set the parm in docker-compose yml file before build container.

https://github.com/doctrine/orm/issues/5622#issuecomment-376401755


⚠ Bofrost.at has degraded performance

In 617fbf4, Bofrost.at (https://www.bofrost.at) experienced degraded performance:

HTTP code: 200
Response time: 1545 ms

Resolved: Bofrost.at performance has improved in a9a2df0 after 3 minutes.


add 4 function to read shp file with Z value

dear sir:
i very thanks that you write this to read shape file, it help me a lot.
when i use this to read 3D(x,y,z) shape file, it just return two dimension x and y, so i read you raw code carefully, and then i write some code to read Z value based on yours.
recently i find some person get in to the same trouble like me, so i want to upload my code, and share to them.
I hope you can accept my code, thanks again.

your follower, wema

Should typenames in shp/polylinez.js be "LineString" or "LineStringZ" ?

i should explain this , there are many types in shape file like las format, so as the definition of ESRI, the type name should be LineStringZ, but for using easier, I set the type name as LineString.

Just a quick comment on this PR.
The GeoJSON spec does not have seperate types for Point and PointZ (and the same for other geoemtry types). If this lib is attempting to align to the geojson spec then bits like this would need some tweaking.
Cheers


FLINK-2166. Add fromCsvFile() method to TableEnvironment. A tuple based

CsvInputFormat, a pojo based CsvReader or a Parameterized tuple based CsvReader is used to represent the row of the table's types. For tuple based CsvInputFormat, a maximum of 25 fields are supported. An CsvOption class is provided for users to set the csv parsing options such as line delimiter, field delimiter etc in the CsvInputFormat based implementation.

Hi @jamescao, thanks for the contribution. I will review your code soon. I'm currently also writing a TableEnvironment fromXXX() method.

Hi @jamescao, thanks for your contribution. I read this pull request and have some comments about this.
First, Test cases are insufficient. There is no test cases to test Scala API, Case Classes, and POJOs. CsvInputFormat supports Tuple, Case Classes, and POJOs. We should add test cases to test them.
Second, I think that Scala implementation is not good to use. There is already logic of reading CSV file in DataSet of Scala API (readCsvFile() method). I think that reusing this method is better then current implementation. readCsvFile() method already supports Case Classes, POJOs and Tuples. The method is tested well. If reuse the method, we can reduce many duplicated codes.
Third, It is just my opinion. I think that test cases using file decreases testing performance. Flink community have tried to improve stability and performance of testing. We are changing test cases to use memory to compare results. collect() method and comparing list or array are sufficient to test. I know there are some test cases using file, but using memory to compare results is better than using file for new test cases.
Fourth, CsvOptions is not same pattern as other configuration object in Flink. In Flink, many configuration objects implement builder pattern. Each setter method should return this object. You can use CsvReader class to reference.

Hi Chiwan
Thanks for your comment! I was new to this community and I am not very familiar with the design conventions.

I was a little bit confused about the api structure. TableEnvironment is in org.apache.flink.api.java.table so I guess it should be a java only api. In that case, the scala test I uploaded is inappropriate in the first place.  If we introduce fromCsvFile for scala table api, it should be in org.apace.flink.api.scala.table.TableEnviroment. Is this right?
I will work on a  scala api using readCsvFile() instead of delegating to the java api.
I will improve the test by comparing the result in memory.
I will make the options comply to the builder pattern.

Thanks!

Just my opinion, TableEnvironment is located under org.apache.flink.api.java.table because of unifying of Table API implementation. But Table API is implemented on Scala. I think that using Scala API is proper for this.
@aljoscha @fhueske How do you think about this?

Maybe we could move the TableEnvironment to org.apache.flink.api.table to make it clear that it can be used for both languages.

+1 to move it

It seems that FLINK-2167 is also facing this api design issue. This will also impact other issues of fromXXX() methods for the table api. I will try to implement a version that move the TableEnvironment to to org.apache.flink.api.table then we can discuss afterward.

It turns out that moving TableEnvironment to org.apache.flink.api.table may not be a clean solution to this problem. the toDataSet() and toDataStream() methods of TableEnvironment will return java DataSet and java DataStream while for scala api, there are already toDataSet and toDataStream  in TableConversions that return correct scala types.
I personally feel that to read csv into a table in scala api, a user could do
val data = env.readCsvFile(...)
Table t = data.toTable()
To combine these two lines in to a fromCsvFile() method doesn't make the task much easier. A lot of duplicated code needs to be writen to pass the csv parsing options and type infos from fromCsvFile() to readCsvFile(). Any advice on this point?
@chiwanpark @fhueske
For the java api,  If we pass CsvReader as an argument to fromCsvFile() or rename it to fromCsvReader(), then we don't need to introduce CsvOptions any more. Which will save a lot of duplicated code in my opinion. Any suggestions?

Hi all, I'm currently working on the HCatInputFormat (FLINK-2167) for the Table API. I think I will complete it this week. That was why I wanted to review the PR later as I have already implemented neccessary API changes. I have already created 2 TableEnviroments one for each API respectively. I'll also introduce a special TableSource interface for input formats.

@twalthr
Great! I will hold my work on this issue. I think FLINK-2167 will set a good reference for other fromXXX issues.

Hi @twalthr, can you give an update for the other Table API InputFormats? Thanks, Fabian

@twalthr @fhueske @vasia Is this still valid with the recent changes in the Table API?

I think this PR has to be rewritten entirely to be in sync with the new Table API and TableDefs which are defined here: https://docs.google.com/document/d/1sITIShmJMGegzAjGqFuwiN_iw1urwykKsLiacokxSw0/edit#heading=h.kdgexc4bw670

Ok, but I gather from you not closing this that you still plan to do the work? :smile:

@aljoscha I don't have the permissions to close it ;-)

Ah, I forgot, it was @jamescao who opened it... :sweat_smile:

This PR can be closed. A CsvTableSource has been implemented with PR #1939.


cargo-make Makefile.toml schema

I put together a schema for cargo-make makefiles.
It works fine with the taplo CLI validation, but there's some quirkiness in the vscode extension, mostly with tooltips for items, where it seems to be pulling every subschema description from oneOf/anyOf schema declarations into the parent schema, and then...making them bold and larger for some reason? Anyway, that appears to be a bug on the vscode extension side, not the schema itself afaik. It might be a good test case for it.
Screenshot of the issue:

This looks great, thanks!
The oneOf/anyOf schemas are not properly handled in the extension right now unfortunately (#57), because they need special care, so instead they're all just merged. I have no idea why they're bold though.


Fresh Raspbian install, RPi3 bridge won't run

@bwssytems - Corey here trying to help someone on my website who is having problem getting the bridge to start.  Fresh Raspbian install, RPi3 (and RPi2) bridge won't run, errors below:
He keeps getting the following error after running: Java -jar /home/pi/echobridge/ha-bridge-3.2.1.jar

Response Port is in use, starting loop to find open port for 20 tries – configured port is: 50000
ERROR spark.webserver.JettySparkServer – ignite failed
java.net.SocketException: Permission denied

Thanks!
—

2016-11-03 09:03:28,009 [main] INFO com.bwssystems.HABridge.HABridge – HA Bridge (v3.2.1) starting….
2016-11-03 09:03:28,079 [main] INFO com.bwssystems.HABridge.BridgeSettings – reading from system properties
2016-11-03 09:03:28,105 [main] INFO com.bwssystems.HABridge.BridgeSettings – Adding <IP_ADDRESS> as our default u pnp config address.
2016-11-03 09:03:28,106 [main] INFO com.bwssystems.HABridge.HABridge – HA Bridge initializing….
2016-11-03 09:03:28,175 [main] INFO com.bwssystems.HABridge.SystemControl – System control service started….
2016-11-03 09:03:28.736:INFO::Thread-0: Logging initialized @2534ms
2016-11-03 09:03:28,925 [main] WARN com.bwssystems.HABridge.dao.DeviceRepository – Error reading the file: data/de vice.db – Does not exist or is not readable. continuing…
2016-11-03 09:03:28,926 [main] INFO com.bwssystems.HABridge.devicemanagmeent.DeviceResource – HABridge device mana gement service started….
2016-11-03 09:03:29,000 [main] INFO com.bwssystems.HABridge.upnp.UpnpSettingsResource – Hue description service st arted….
2016-11-03 09:03:29,014 [main] INFO com.bwssystems.util.UDPDatagramSender – Initializing UDP response Seocket…
2016-11-03 09:03:29,028 [main] WARN com.bwssystems.util.UDPDatagramSender – UDP Response Port is in use, starting loop to find open port for 20 tries – configured port is: 50000
2016-11-03 09:03:29,029 [main] INFO com.bwssystems.util.UDPDatagramSender – UDP response Seocket initialized to: 5 0001
2016-11-03 09:03:29,199 [Thread-0] INFO spark.webserver.JettySparkServer – == Spark has ignited …
2016-11-03 09:03:29,209 [Thread-0] INFO spark.webserver.JettySparkServer – >> Listening on <IP_ADDRESS>:80
2016-11-03 09:03:29.226:INFO:oejs.Server:Thread-0: jetty-9.3.z-SNAPSHOT
2016-11-03 09:03:29,463 [Thread-0] ERROR spark.webserver.JettySparkServer – ignite failed
java.net.SocketException: Permission denied
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:433)
at sun.nio.ch.Net.bind(Net.java:425)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:326)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:244)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:384)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at spark.webserver.JettySparkServer.ignite(JettySparkServer.java:131)
at spark.SparkInstance.lambda$init$0(SparkInstance.java:341)
at java.lang.Thread.run(Thread.java:745)

User was using Raspbian LITE and/or Raspbian installed by NOOBS LITE.  Need full Raspbian to run ha-bridge.  Also finding that port 80 is requiring sudo, flushing this out now...so closing Issue.

Lite just has less pre-installed. Did you ever determine what was needed to overcome the errors beyond installing a new image of Raspbian?

BUMP.
I just installed jessie (just checked by cat /etc/os-release) and have this same exact problem.  No where in this file does it say "lite".  So I am stumped.  Especially since I just went through Corey's how-to just last month.  (I trashed my SDCard and had to start over).  Aside from possible pilot error, the only thing I didn't do was start with Corey's old how-to then proceed to his new how-to.  Maybe there is something missing by skipping over doing Corey's old how-to.

Sometimes it is best to go with the standard Raspbian instructions and go from there.

Yes, I understand what you are saying.  I am fairly sure that this (https://www.raspberrypi.org/downloads/raspbian/) is what I downloaded and installed.  The raspberry pi version of Jessie.  "Raspbian Jessie with PIXEL".

are you running the bridge with sudo or giving port 80 access to the pi user?

Linux.  I used "dd".  So I assume the SDCard was erased in the process.  The SDCard is 16GBytes.

No, I believe dd does not erase the sd card when writing the image.  You can erase an sd card with dd using the following: sudo dd if=/dev/zero of=/dev/disk2 bs=8192, being SUPER sure you have the correct disk in the "of=" part, I used disk2 as this was my disk's name.  You can also use the fdisk or cfdisk command to erase the card.   Once erased, the card will be unmounted and you can dd the image on thereafter.

BTW, there's a difference in the startbridge.sh script.  Corey has:
nohup java -jar -Djava.net.preferIPv4Stack=true /home/pi/habridge/ha-bridge.jar > /home/pi/habridge/habridge-log.txt 2>&1 &
...and here it's:
nohup java -jar -Dconfig.file=/home/pi/habridge/data/habridge.config /home/pi/habridge/ha-bridge-3.5.1.jar > /home/pi/habridge/habridge-log.txt 2>&1 &
...I'm not sure if this is a small update or something important.
(Thanks for the help guys.)

Ok, @Corey, I'll start over.  Check the MD5's this time too.
-thanks.

-Djava.net.preferIPv4Stack=true is actually now not required, as it is default.
I name my jar file simply "ha-bridge.jar" so I don't have to edit the startbridge.sh script.  I simply download the latest, rename it, and reboot.
So essentially the two are the same, I'll remove the -Djava.net.preferIPv4Stack=true from my tutorial now.

Ok, it worked.  The SHA1 check good on the original raspbean file so I used it again.  Maybe dd doesn't wipe a SDCard clean.  But I am going to side with me pushing the install.  I think I executed a config  or make early before the apt-get upgrade was finished.


Error thrown with correct query

import { createBaseTable, orchidORM } from "orchid-orm";

export const BaseTable = createBaseTable({
    columnTypes: t => ({
        ...t,
        text: (min = 0, max = 1024) => t.text(min, max),
    })
});

export class AccountsTable extends BaseTable {
    table = "accounts";

    columns = this.setColumns(t => ({
        id: t.text().primaryKey(),
        public_key: t.text(),
        name: t.text(),
        alias: t.text().nullable(),
        bio: t.text(),
        avatar_storage_id: t.text().nullable(),
        avatar_etag: t.text().nullable(),
        created_at: t.timestamp(),
        updated_at: t.timestamp(),
        last_seen_at: t.timestamp(),
        is_verified: t.boolean(),
    }));

    relations = {
        cover: this.hasOne(() => AccountCoversTable, {
            primaryKey: "id",
            foreignKey: "account_id",
        }),
        oauths: this.hasMany(() => AccountOauthsTable, {
            primaryKey: "id",
            foreignKey: "account_id",
        }),
        account_posts: this.hasMany(() => PostsTable, {
            primaryKey: "id",
            foreignKey: "author_id",
        }),
        account_likes: this.hasMany(() => LikesOnPostsTable, {
            primaryKey: "id",
            foreignKey: "liker_id",
        }),
    };
}

export class AccountCoversTable extends BaseTable {
    table = "account_covers";

    columns = this.setColumns(t => ({
        account_id: t.text().foreignKey("accounts", "id").primaryKey(),
        storage_id: t.text(),
        etag_id: t.text(),
        created_at: t.timestamp(),
        updated_at: t.timestamp(),
    }));
}

export class AccountOauthsTable extends BaseTable {
    table = "account_oauths";

    columns = this.setColumns(t => ({
        account_id: t.text().foreignKey("accounts", "id").primaryKey(),
        provider_id: t.enum("oauth_provider_type", ["instagram", "twitter"]),
        user_id: t.text(),
        username: t.text(),
    }));
}

export class PostsTable extends BaseTable {
    table = "posts";

    columns = this.setColumns(t => ({
        id: t.text().primaryKey(),
        author_id: t.text().foreignKey("accounts", "id"),
        content: t.text(),
        created_at: t.timestamp(),
        updated_at: t.timestamp(),
    }));

    relations = {
        comments: this.hasMany(() => CommentsOnPostsTable, {
            primaryKey: "id",
            foreignKey: "post_id",
        }),
        likes: this.hasMany(() => LikesOnPostsTable, {
            primaryKey: "id",
            foreignKey: "post_id",
        }),
    };
}

export class CommentsOnPostsTable extends BaseTable {
    table = "comments_on_posts";

    columns = this.setColumns(t => ({
        post_id: t.text().foreignKey("posts", "id").primaryKey(),
        comment_id: t.text().foreignKey("comments", "id"),
    }));
}

export class LikesOnPostsTable extends BaseTable {
    table = "likes_on_posts";

    columns = this.setColumns(t => ({
        liker_id: t.text().foreignKey("accounts", "id").primaryKey(),
        post_id: t.text().foreignKey("posts", "id").primaryKey(),
        created_at: t.timestamp(),
    }));
}

export const db = orchidORM({ databaseURL: process.env["DATABASE_URL"], autoPreparedStatements: true, log: true }, {
    accounts: AccountsTable,
    accountCovers: AccountCoversTable,
    accountOauths: AccountOauthsTable,
    posts: PostsTable,
    commentsOnPosts: CommentsOnPostsTable,
    LikesOnPosts: LikesOnPostsTable,
});

const results = db.accounts
    .leftJoin("cover")
    .selectAll()
    .select({
        cover: q => q.cover.selectAll(),
        num_posts: q => q.account_posts.count().whereNotExists(db.commentsOnPosts, "comment_id", "=", "id"),
        num_comments: q => q.account_posts.count().whereExists(db.commentsOnPosts, "comment_id", "=", "id"),
        num_posts_liked: q => q.account_likes.count().whereNotExists(db.commentsOnPosts, "comment_id", "=", "post_id"),
        num_comments_liked: q => q.account_likes.count().whereExists(db.commentsOnPosts, "comment_id", "=", "post_id"),
    })
    .limit(10);

console.log(await results);

Calling await at the end yields an arbitrary error:
% node --no-warnings --loader ts-node/esm src/db.orchid.ts
(39.7ms) SELECT "accounts".*, (SELECT row_to_json("t".*) FROM (SELECT * FROM "account_covers" AS "cover" WHERE "cover"."account_id" = "accounts"."id" LIMIT $1) AS "t") AS "cover", (SELECT count(*) FROM "posts" AS "account_posts" WHERE NOT EXISTS (SELECT 1 FROM "comments_on_posts" WHERE "comments_on_posts"."comment_id" = "account_posts"."id" LIMIT 1) AND "account_posts"."author_id" = "accounts"."id") AS "num_posts", (SELECT count(*) FROM "posts" AS "account_posts" WHERE EXISTS (SELECT 1 FROM "comments_on_posts" WHERE "comments_on_posts"."comment_id" = "account_posts"."id" LIMIT 1) AND "account_posts"."author_id" = "accounts"."id") AS "num_comments", (SELECT count(*) FROM "likes_on_posts" AS "account_likes" WHERE NOT EXISTS (SELECT 1 FROM "comments_on_posts" WHERE "comments_on_posts"."comment_id" = "account_likes"."post_id" LIMIT 1) AND "account_likes"."liker_id" = "accounts"."id") AS "num_posts_liked", (SELECT count(*) FROM "likes_on_posts" AS "account_likes" WHERE EXISTS (SELECT 1 FROM "comments_on_posts" WHERE "comments_on_posts"."comment_id" = "account_likes"."post_id" LIMIT 1) AND "account_likes"."liker_id" = "accounts"."id") AS "num_comments_liked" FROM "accounts" LEFT JOIN "account_covers" AS "cover" ON "cover"."account_id" = "accounts"."id" LIMIT $2 [1, 10]
file:///Users/dew/node_modules/pqb/src/queryMethods/then.ts:41
    temporaryError = new Error();
                     ^
Error
    at Db.get then [as then] (file:///Users/dew/node_modules/pqb/src/queryMethods/then.ts:41:22)
    at file:///Users/dew/src/db.orchid.ts:139:19
    at ModuleJob.run (node:internal/modules/esm/module_job:193:25)
    at async Promise.all (index 0)
    at async ESMLoader.import (node:internal/modules/esm/loader:530:24)
    at async loadESM (node:internal/process/esm_loader:91:5)
    at async handleMainPromise (node:internal/modules/run_main:65:12)

The query alone however is built correctly and works should I manually execute it in my PostgreSQL database.

Thank you for catching it, I'll be able to test and fix today's evening or tomorrow.

The table for comments is missing:
export class CommentsOnPostsTable extends BaseTable {
    table = "comments_on_posts";

    columns = this.setColumns(t => ({
        post_id: t.text().foreignKey("posts", "id").primaryKey(),
        comment_id: t.text().foreignKey("comments", "id"), // <-- here
    }));
}

I'll make an update for the error message to indicate this clearly.

Why ids are of text type, by the way, is there anything wrong with uuid type?
I guess it's uuid in your database, but you have used text just for quick testing.

The bug is fixed now, error reporting is improved as well.
@lithdew please confirm when you'll have time.
I've found various issues on my way to reproduce and debug it, added and fixed various things. Added support for enum (previously missing) to the migrations, fixed parsing of the subquery, so now if it is a count it will be returned as a number, previously wasn't parsed and was returned as a string.
I hope won't give up on it after a bug or two, your feedback really helps!

By the way, .leftJoin("cover") in the query has no effect because LEFT JOIN, unlike just JOIN, doesn't require the related record to exist, and it's not used in select or where.
q.cover.selectAll(), - returning all columns is a default, so it can be simplified to q.cover.
.whereNotExists(db.commentsOnPosts, "comment_id", "=", "id") also can be written as:
.whereNotExists(db.commentsOnPosts, "comment_id", "id")
// or
.whereNotExists(db.commentsOnPosts, { comment_id: "id" })

fixed parsing of the subquery, so now if it is a count it will be returned as a number, previously wasn't parsed and was returned as a string.

There's a catch here. The count returned by PostgreSQL is a bigint, and can therefore be larger than the max integer value.
9223372036854775807 // PostgreSQL bigint
9007199254740991 // Number.MAX_SAFE_INTEGER

This is the reason node-pg returns counts as strings, but it's the last thing anyone expects so is the source of lots of bugs. I'm not sure what a good answer is, but maybe the most correct solution would be to return a big-number or equivalent

I believe the maximum<PHONE_NUMBER>740991 is more than enough for counting records, and that returning a bigint or a string would only cause trouble.
JS has bigint type by the way:
typeof<PHONE_NUMBER>854775807n === 'bigint'

but it's not a number and it can't do arithmetic with numbers, only after casing numbers to bigint explicitly.

You're probably right. Just to be on the safe side, maybe you could report an error if the count was ever bigger than Number.MAX_SAFE_INTEGER. Otherwise if anyone was unfortunate enough to exceed the limit, an incorrect count would be returned.


fix: Remove duplicate "command" in post-install-hook

Hi @svmaris  and thank you for noticing.
We are removing the job's from the helm installation so I will not merge this PR :)


error about azcopy

Hello! When I run
AZFUSE_TSV_USE_FUSE=1 python -m generativeimage2text.inference -p "{'type': 'test_git_inference_single_image', \
      'image_path': 'aux_data/images/1.jpg', \
      'model_name': 'GIT_BASE', \
      'prefix': '', \
}"

The following error occurred. I'm not sure what the reason is, whether it was an error installing azfuse. If you could provide suggestions, I would greatly appreciate it.

as your log shows, the error is following:
permission denied: /home/xxx/azcopy.
Thus,
1.check if this azcopy file path can correctly localize your azcopy?
2. run the command tool and see if it is executible. make sure -x added to the azcopy file. (chmod +x /home/xxx/azcopy)


go mod tidy error

get https://sourcegraph.com/sourcegraph/appdash?go-get=1
get https://sourcegraph.com/sourcegraph/appdash?go-get=1: 404 Not Found (0.369s)
get https://sourcegraph.com/sourcegraph/appdash?go-get=1
get https://sourcegraph.com/sourcegraph/appdash?go-get=1: 404 Not Found (0.203s)
go<EMAIL_ADDRESS>requires
<EMAIL_ADDRESS>unrecognized import path "sourcegraph.com/sourcegraph/appdash": reading https://sourcegraph.com/sourcegraph/appdash?go-get=1: 404 Not Found

Hi Iwan, can you provide us with more context for this issue? Did you encounter this while installing/updating GMS?
I ran go get<EMAIL_ADDRESS>at HEAD and was not able to reproduce this issue.

Hi Iwan, we haven't heard back in a month on this issue, so I'm going to close it out. Feel free to reopen and provide the information Nick asked for if you're still seeing this issue.


Added uranium server support

Added Uranium server support

@xjboss would you mind linking to the server software's official website?

Some lurking leaded me to this: https://github.com/UraniumMC/Uranium


Error: write after end

I wrote a gulp task using this article:
var browserified = transform(function(filename) {
    var b = browserify(filename);
    b.transform('brfs');
    return b.bundle();
});

return gulp.src(src)
    .pipe(browserified)
    .pipe(uglify())
    .pipe(gulp.dest(dest));

When I run this, I get an error: Error: write after end     at writeAfterEnd (/www/my-project/node_modules/browserify/node_modules/labeled-stream-splicer/node_modules/stream-splicer/node_modules/readable-stream/lib/_stream_writable.js:161:12)
If I comment out the  b.transform('brfs');, everything works fine.  Does anyone have any insight as to why this is happening?
Thanks!

Which versions of browserify and brfs? None of the above two ways work with browserify 8.1.0 + brfs 1.3.0 (same error as above happens).

I'm using:
"brfs": "^1.2.0",
"browserify": "^7.0.2",

Looks like I shouldn't upgrade...

Just try woth browserify 8.X.X, you'll see :)

Please, reopen this issue, it is not fixed at all. It fails with browserify 8.X.X.

Still this bug? Neither bundler.transform('brfs') or bundler.transform(require('brfs')) works:
Error: write after end
    at writeAfterEnd (/Users/ibc/eFace2Face/src/DoctoRTCWeb/node_modules/browserify/node_modules/readable-stream/lib/_stream_writable.js:161:12)
    at Labeled.Writable.write (/Users/ibc/eFace2Face/src/DoctoRTCWeb/node_modules/browserify/node_modules/readable-stream/lib/_stream_writable.js:208:5)
    at Producer.ondata (_stream_readable.js:540:20)
    at Producer.emit (events.js:107:17)
    at Producer.Readable.read (_stream_readable.js:373:10)
    at flow (_stream_readable.js:750:26)
    at resume_ (_stream_readable.js:730:3)
    at _stream_readable.js:717:7
    at process._tickDomainCallback (node.js:381:11)

The lack of comments from the developer about this issue is frustrating. I know more people with the same issue.
My versions:

browserify 9.0.3
brfs 1.4.0

I don't know the state of the API; everything I tried fails with the same error. I eventually switched to using the CLI, which does seem to work for some obnoxious reason. I'd love to see the API working again as well, or some reaction from the maintainer, but @ibc you might be able to use this meanwhile?

Looks like this is an ongoing issue with gulp/browserify. I was able to workaround this by using through2 instead of transform (the solution listed here).

but @ibc you might be able to use this meanwhile?

I've been able to use gulp-brfs and, somehow, fills my needs.

b.bundle() returns a readable stream, not a transform stream. Many of the gulp guides are mistaken on this point but are slowly being fixed. Here's a good fix by @terinjokes for gulp users: https://github.com/substack/node-browserify/issues/1191#issuecomment-89302289


Is is possible to run the pre-trained model with rtx 2060 6g?

My computer has the rtx 2060 6G, the results say my gpu is lack, is there any way to run the phoenix model. thank you.

Hi @dream-desktop,
Thanks for your attention. Currently, 6G is not sufficient for running Phoenix. We will work on the Quantization version.
Or you can try Phoenix using cpu:
python -m llmzoo.deploy.cli --model-name FreedomIntelligence/phoenix-inst-chat-7b --device cpu

which requires ~30GB of CPU memory.
Best,
Zhihong

Please check if this helps: https://gist.github.com/rain-1/8cc12b4b334052a21af8029aa9c4fafc
@dream-desktop

It is for LLaMA based models


What is the difference between a check-in and the main answer of a template?

#60 #31 are concerned by this question.
We have mapped out answering a template after you've created it, and also setting up check-ins to see how your goals are going. What is the difference between these in terms of questions, but also progress. Do you have to specifically set an end-date to your template/goal where you have to fill out the whole thing again, or can you choose to do so on every check-in, should you want to?

After you've finished building your template, you will get to a page where you can specify your check-in goals. Here you'll answer how frequently you want to fill in a check-in, and more importantly specify which questions will be asked. It's likely this will be a smaller subset of the initial questions.
The initial template will have to have some sort of 'end date' on it, so that you have something to work towards, and the full template will only ever be completed at the beginning & end of that.


onDone- Call back not calling

onDone- Call back not calling and not working for our test server url
Always getting "Error is The operation couldn’t be completed. (internet_speed_test.NetworkError error 0.)"
I tested in iOS only

same problem


add bench setup.py command

Use the benchmark test plugin

This is now done.
python setup.py bench -l


Riot crashes when selecting Silent
as notification sound

Should not be long to fix

Caused by: kotlin.TypeCastException: null cannot be cast to non-null type android.net.Uri
        at im.vector.fragments.VectorSettingsPreferencesFragment.onActivityResult(VectorSettingsPreferencesFragment.kt:1243)


sshd example with UserSwitch

Expected/Desired Behavior:
I'm trying to get the sshd.py example to work with a userspace switch. I'm expecting the startup process to run until I get a mininet> shell and to be able to connect onto the hosts through SSH. I'm getting the expected result if I use the default Open vSwitch.
Actual Behavior:
The startup process remains stuck on *** Waiting for ssh daemons to start as it cannot join the hosts through SSH. Even with a separate shell, I cannot connect to any of the hosts on port 22, although I can ping all of them successfully.
Detailed Steps to Reproduce the Behavior:
I changed sshd.py to use the userspace switch (the default OF1.3 UserSwitch) and a remote controller (in my case, Floodlight):
diff --git a/examples/sshd.py b/examples/sshd.py
index 7e0f517..fcf7a16 100755
--- a/examples/sshd.py
+++ b/examples/sshd.py
@@ -21,7 +21,7 @@
 from mininet.net import Mininet
 from mininet.cli import CLI
 from mininet.log import lg, info
-from mininet.node import Node
+from mininet.node import Node, UserSwitch, RemoteController
 from mininet.topolib import TreeTopo
 from mininet.util import waitListening

@@ -29,7 +29,7 @@
 def TreeNet( depth=1, fanout=2, **kwargs ):
     "Convenience function for creating tree networks."
     topo = TreeTopo( depth, fanout )
-    return Mininet( topo, **kwargs )
+    return Mininet( topo, controller=lambda a: RemoteController( a, ip="<IP_ADDRESS>", port=6653 ), switch=UserSwitch, *kwargs )

 def connectToRootNS( network, switch, ip, routes ):
     """Connect hosts to root namespace via switch. Starts network.

When I run the script, it blocks on the SSH connections:
$ sudo python sshd.py
*** Creating network
*** Adding controller
*** Adding hosts:
h1 h2 h3 h4
*** Adding switches:
s1
*** Adding links:
(s1, h1) (s1, h2) (s1, h3) (s1, h4)
*** Configuring hosts
h1 h2 h3 h4
*** Starting controller
c0
*** Starting 1 switches
s1
*** Waiting for ssh daemons to start
^CTraceback (most recent call last):
  File "sshd.py", line 84, in <module>
    sshd( net, opts=argvopts )
  File "sshd.py", line 66, in sshd
    waitListening( server=server, port=22, timeout=5 )
  File "build/bdist.linux-x86_64/egg/mininet/util.py", line 606, in waitListening
  File "build/bdist.linux-x86_64/egg/mininet/util.py", line 133, in quietRun
  File "build/bdist.linux-x86_64/egg/mininet/util.py", line 96, in errRun
KeyboardInterrupt

Additional Information:
I checked with ip a that all network interfaces have been successfully created and with top that the sshd processes are waiting.

I had also the same issue. and for me, i just start ssh service using :
systemctl start ssh


[SPARK-30298][SQL] Respect aliases in output partitioning of projects and aggregates

What changes were proposed in this pull request?

Currently, in the following scenario, bucket join is not utilized:
val df = (0 until 20).map(i => (i, i)).toDF("i", "j").as("df")
df.write.format("parquet").bucketBy(8, "i").saveAsTable("t")
sql("CREATE VIEW v AS SELECT * FROM t")
sql("SELECT * FROM t a JOIN v b ON a.i = b.i").explain

== Physical Plan ==
*(4) SortMergeJoin [i#13], [i#15], Inner
:- *(1) Sort [i#13 ASC NULLS FIRST], false, 0
:  +- *(1) Project [i#13, j#14]
:     +- *(1) Filter isnotnull(i#13)
:        +- *(1) ColumnarToRow
:           +- FileScan parquet default.t[i#13,j#14] Batched: true, DataFilters: [isnotnull(i#13)], Format: Parquet, Location: InMemoryFileIndex[file:..., PartitionFilters: [], PushedFilters: [IsNotNull(i)], ReadSchema: struct<i:int,j:int>, SelectedBucketsCount: 8 out of 8
+- *(3) Sort [i#15 ASC NULLS FIRST], false, 0
   +- Exchange hashpartitioning(i#15, 8), true, [id=#64] <----- Exchange node introduced
      +- *(2) Project [i#13 AS i#15, j#14 AS j#16]
         +- *(2) Filter isnotnull(i#13)
            +- *(2) ColumnarToRow
               +- FileScan parquet default.t[i#13,j#14] Batched: true, DataFilters: [isnotnull(i#13)], Format: Parquet, Location: InMemoryFileIndex[file:..., PartitionFilters: [], PushedFilters: [IsNotNull(i)], ReadSchema: struct<i:int,j:int>, SelectedBucketsCount: 8 out of 8

Notice that Exchange is present. This is because Project introduces aliases and outputPartitioning and requiredChildDistribution do not consider aliases while considering bucket join in EnsureRequirements. This PR addresses to allow this scenario.
Why are the changes needed?

This allows bucket join to be utilized in the above example.
Does this PR introduce any user-facing change?

Yes, now with the fix, the explain out is as follows:
== Physical Plan ==
*(3) SortMergeJoin [i#13], [i#15], Inner
:- *(1) Sort [i#13 ASC NULLS FIRST], false, 0
:  +- *(1) Project [i#13, j#14]
:     +- *(1) Filter isnotnull(i#13)
:        +- *(1) ColumnarToRow
:           +- FileScan parquet default.t[i#13,j#14] Batched: true, DataFilters: [isnotnull(i#13)], Format: Parquet, Location: InMemoryFileIndex[file:.., PartitionFilters: [], PushedFilters: [IsNotNull(i)], ReadSchema: struct<i:int,j:int>, SelectedBucketsCount: 8 out of 8
+- *(2) Sort [i#15 ASC NULLS FIRST], false, 0
   +- *(2) Project [i#13 AS i#15, j#14 AS j#16]
      +- *(2) Filter isnotnull(i#13)
         +- *(2) ColumnarToRow
            +- FileScan parquet default.t[i#13,j#14] Batched: true, DataFilters: [isnotnull(i#13)], Format: Parquet, Location: InMemoryFileIndex[file:.., PartitionFilters: [], PushedFilters: [IsNotNull(i)], ReadSchema: struct<i:int,j:int>, SelectedBucketsCount: 8 out of 8

Note that the Exchange is no longer present.
How was this patch tested?

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20341/
Test PASSed.

Test build #115540 has started for PR 26943 at commit d3559b4.

Test build #115540 has finished for PR 26943 at commit d3559b4.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115540/
Test FAILed.

retest this please

Test build #115545 has started for PR 26943 at commit d3559b4.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20348/
Test PASSed.

Test build #115545 has finished for PR 26943 at commit d3559b4.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115545/
Test FAILed.

retest this please

Test build #115576 has started for PR 26943 at commit d3559b4.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20376/
Test PASSed.

Test build #115576 has finished for PR 26943 at commit d3559b4.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/115576/
Test PASSed.

Have you checked SPARK-25951, SPARK-19981, and  SPARK-19468? I think the other project-like operators (e.g., aggregate) has the same issue.

Thanks @maropu for the info! Adding the same outputPartitioning logic to HashAggregateExec correctly handles cases reported in #22957 and #17400. I can make this more generic similar to what was proposed in those PRs. But could you explain why those two PRs never got merged?

I think we just couldn't reach a consensus about how-to-fix for the issue.

I see. @maropu Do you mind if I take your ideas from #17400 like AliasAwareOutputPartitioning and put them here? I would like to try one more time with the fix if possible.

yea, sure! You can feel free to take them over.

Test build #116062 has started for PR 26943 at commit c24789d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20854/
Test PASSed.

@cloud-fan @gatorsmile @viirya This addresses the same issues brought up in #22957 and #17400. I understand those two PRs didn't get merged, but wanted to give another shot at it. (We had few customers asking why bucket join was not respected when aliases were used). Could you help reviewing? Thanks in advance!

Test build #116062 has finished for PR 26943 at commit c24789d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait AliasAwareOutputPartitioning extends UnaryExecNode

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116062/
Test FAILed.

retest this please

Test build #116066 has started for PR 26943 at commit c24789d.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20858/
Test PASSed.

Test build #116066 has finished for PR 26943 at commit c24789d.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
trait AliasAwareOutputPartitioning extends UnaryExecNode

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116066/
Test PASSed.

looks fine to me. Shall we also consider some corner cases like a + 1 as b and the child's output partitioning is hash(a + 1)?

Test build #116197 has started for PR 26943 at commit fcd2186.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/20990/
Test PASSed.

Test build #116198 has started for PR 26943 at commit 1762a96.

Test build #116197 has finished for PR 26943 at commit fcd2186.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116197/
Test PASSed.

Test build #116198 has finished for PR 26943 at commit 1762a96.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/116198/
Test PASSed.

I'll check later and thanks for the rework, @imback82!  cc: @mgaido91

@imback82 I think outputOrdering has the same issue and do you wanna work on that, too?
#17400 (comment)

Yes, I will work on it.

@imback82 You're still working on this?

@imback82 You're still working on that?

Yes! I tried the failing example in the comment, but that was working fine in the latest version. I will look into this further.

yea, thanks!

@maropu I created a PR for outputOrdering: https://github.com/apache/spark/pull/27842


CLI: Pre-release dependency audit

What problem does your feature solve?
Before releasing to the public we should go through all our dependencies, checking for potential security & licensing issues.
What would you like to see?
Maybe a list of all the dependencies with a security, code quality, and licensing info?
What alternatives are there?

@paulbellamy Is this still critical and for what reason given it hasn't been planned for an iteration yet.
Should we have this included in iteration prior to Meridian?


Switch to nginx cookbook for testing purposes

Maintained & up-to-date
Functionally equivalent to chef_nginx
Needs a workaround for CentOS/RHEL systems

@thoutenbos, I think this will fix the builds we found broken on #90. Works better on my system!

@thoutenbos I got it! The build is now only failing b/c of a single foodcritic thing, which I left unchanged due to my lack of depth on the cookbook.

Excellent, many thanks! Still don't fully understand how this could cause the remote_file failures but happy to have the build working again 👍


Study locus page error/crashes when there are no results

When the study locus page is loaded with a URL that has no data, it crashes instead of returning empty tables.
For example an unknown study:
https://genetics.opentargets.org/study-locus/UNKNOWN/1_109274968_G_T
In this case the frontend will return the following error:
TypeError: "v is null"
    value StudyLocusPage.js:339

Or when a variant is loaded without having L2G data, for example:
https://genetics.opentargets.org/study-locus/GCST002222/1_109274968_G_A
If a variant with no data is loaded, the following frontend error will be returned:
TypeError: "j is null"
    value StudyLocusPage.js:391

My guess is that somewhere, the GraphQL error handling is not done properly on this page.
Thanks!

thanks @BjornWouters  for filling this issue. From the api, it comes as null and it is a probable case. Regarding the UNKNOWN, may I know how you got to that url? thanks!

Thanks for the fast response @mkarmona. The uknown study identifier was just used as an example, there are no studies in Open Targets Genetics having that identifier.
Even though it is an unlikely situation that a study does not contain L2G data, I think handling a error for that would increase the UX for that page.

@BjornWouters, we just had a quick chat about this during our weekly team meeting and @mirandaio will investigate further
cc @Sjoerd-van-Hagen

Ticket has been removed from the v6 release; instead, it will be part of a broader refactor to implement error boundaries to catch errors without crashing the entire application.

work completed


subsys: fcb: Check for mutex lock failure when walking FCB

Coverity-CID: 182195
Signed-off-by: Leandro Pereira<EMAIL_ADDRESS>

Codecov Report

Merging #6248 into master will decrease coverage by <.01%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #6248      +/-   ##
==========================================
- Coverage   53.11%   53.11%   -0.01%
==========================================
  Files         414      414
  Lines       40260    40260
  Branches     7759     7759
==========================================
- Hits        21385    21384       -1
- Misses      15708    15709       +1
  Partials     3167     3167

Impacted Files
Coverage Δ

samples/philosophers/src/main.c
96.87% <0%> (-1.57%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b2434f...9ead947. Read the comment docs.