SSH_LISTEN_HOST accepts only one address

Gitea version (or commit ref): 1.12.2
Git version: 2.27.0
Operating system: FreeBSD 12.1-RELEASE-p7 amd64
Database (use [x]):

[ ] PostgreSQL
[ ] MySQL
[ ] MSSQL
[x] SQLite

Can you reproduce the bug at https://try.gitea.io:

[ ] Yes (provide example URL)
[ ] No
[x] Not relevant

Log gist:

Description
I have a host with multiple interfaces and addresses. Therefore I'd like to have the dedicated SSH daemon listen on only two particular addresses. In my case a IPv4 and a IPv6 address.
The configuration parameter SSH_LISTEN_HOST does not seem to accept more than one address and defining it on a second line will overwrite the first one.
I tried the following combinations:
Try 1:
SSH_LISTEN_PORT = 2222
SSH_LISTEN_HOST = <IP_ADDRESS> [2001:0db8:85a3:08d3:1319:8a2e:0370]

Result try 1:
2020/08/03 17:12:09 ...s/graceful/server.go:79:ListenAndServe() [E] Unable to GetListener: address <IP_ADDRESS> [2001:0db8:85a3:08d3:1319:8a2e:0370]:2222: too many colons in address
2020/08/03 17:12:09 .../ssh/ssh_graceful.go:23:listen() [F] Failed to start SSH server: address <IP_ADDRESS> [2001:0db8:85a3:08d3:1319:8a2e:0370]:2222: too many colons in address

Try 2:
SSH_LISTEN_PORT = 2222
SSH_LISTEN_HOST = <IP_ADDRESS>,[2001:0db8:85a3:08d3:1319:8a2e:0370]

Result:
2020/08/03 17:13:56 ...s/graceful/server.go:79:ListenAndServe() [E] Unable to GetListener: address <IP_ADDRESS>,[2001:0db8:85a3:08d3:1319:8a2e:0370]:2222: too many colons in address
2020/08/03 17:13:56 .../ssh/ssh_graceful.go:23:listen() [F] Failed to start SSH server: address <IP_ADDRESS>,[2001:0db8:85a3:08d3:1319:8a2e:0370]:2222: too many colons in address

Try 3:
SSH_LISTEN_PORT = 2222
SSH_LISTEN_HOST = <IP_ADDRESS>, [2001:0db8:85a3:08d3:1319:8a2e:0370]

Result:
2020/08/03 17:23:24 ...s/graceful/server.go:79:ListenAndServe() [E] Unable to GetListener: address <IP_ADDRESS>, [2001:0db8:85a3:08d3:1319:8a2e:0370]:2222: too many colons in address
2020/08/03 17:23:24 .../ssh/ssh_graceful.go:23:listen() [F] Failed to start SSH server: address <IP_ADDRESS>, [2001:0db8:85a3:08d3:1319:8a2e:0370]:2222: too many colons in address

Try 4:
SSH_LISTEN_PORT = 2222
SSH_LISTEN_HOST = <IP_ADDRESS>; [2001:0db8:85a3:08d3:1319:8a2e:0370];

Result:
Will only listen to <IP_ADDRESS> but not 2001:0db8:85a3:08d3:1319:8a2e:0370

Workaround: comment SSH_LISTEN_HOST so the gitea SSH daemon will listen on every device and firewall undesired access.

Have reproduced, but dived deeper.
I've created my gitea.yml setup, made a lot of changes and... Just discovered that if you have existing docker volume, changes made to .yml file won't take effect.

Make config file and specify SSH options, you must use docker volume to store your data
Build and start container
Change options in .yml file
Destroy all docker data but don't touch existing volume
rebuild and start container
You'll see that making changes in .yml file doesn't take any effect

Cheers and please fix it asap, as it's impossible to :

change configuration in .yml file, you'll need to somehow get into container that have mounted used volume and change .ini file by hand
it's hard to recover persisted data with different configuration while making changes to yml file
even changing domain woudn't have any effect

Maybe it's only related to ssh options, haven't tested that deep. Anyway by removing volume which result in data loss changes in yml file have effect.

Please change labels as it is not enchancement/proposal but a bug, which exist in a long time. @lunny
Sorry for mentioning, but i think this bug shouldn't be lost in 1.8k others, as docker is today de facto standard.

Not everyone uses Docker. I use NixOS. There's no such thing as any one "de facto" standard, but that is imho and not relevant for this issue. And I do have the same issue: I want to explicitly specify which hosts (IPv4 and IPv6) the SSH server should listen on, because my system has many IPv4 and IPv6 addresses.


CheckTypeSize fails when running in MSYS2

Describe the bug
The following fairly trivial SConstruct file fails on MSYS2:
# SConstuct
import os
env = Environment(tools=["default","mingw"])
conf = Configure(env)
conf.env['SIZEOF_INT'] = conf.CheckTypeSize("int")
conf.env['SIZEOF_VOIDP'] = conf.CheckTypeSize("void *")
env = conf.Finish()
print("size of int:",env['SIZEOF_INT'])
print("size of void*:",env['SIZEOF_VOIDP'])
print("env['PLATFORM'] =",env['PLATFORM'])
print("env['HOST_OS'] =",env['HOST_OS'])

env.Program('hello',['hello.c'])

// hello.c
#include<stdio.h>
int main(void){fprintf(stderr,"hello from C\n"); return 0;}

The output is below. The output '.sconf_temp' is not recognized comes from Windows being passed a forward-slash-separated path, because on this version of Python, os.path.sep is set to /.
john@WIN-HG6RBDGB9LV UCRT64 ~/testsize
$ scons --config=force
scons: Reading SConscript files ...
'.sconf_temp' is not recognized as an internal or external command,
operable program or batch file.
Checking size of int ... no
'.sconf_temp' is not recognized as an internal or external command,
operable program or batch file.
Checking size of void * ... no
size of int: 0
size of void*: 0
env['PLATFORM'] = win32
env['HOST_OS'] = win32
scons: done reading SConscript files.
scons: Building targets ...
scons: `.' is up to date.
scons: done building targets.

A solution for this issue would be the automatic detection of the MSYS platform, (issue #4311), but there may be other simpler fixes to consider.
Required information

Discord discussion: https://discord.com/channels/571796279483564041/1078868924248105032
Version of SCons: 4.5.1
Version of Python: 3.10.10 (mingw-w64-ucrt-x86_64-python 3.10.10-1 in MSYS2)
Which python distribution if applicable: MSYS2
How you installed SCons: pip install scons
What Platform are you on? MSY2
How to reproduce your issue? As above.
How you invoke scons scons --config=force

The workaround for this issue is to invoke MSYSTEM= scons, which disables MSYS2's Python from setting os.path.sep to /.


Error when running mbImpute

Hello, mbImpute developers,
I read your mbImpute manuscript on bioRxiv recently. I really liked the idea and would like to give it a try on my datasets. However I kept getting the following error when running the imputation:

This is the code I'm using:
a_imp <- mbImpute(condition = a_meta_num, otu_tab = otu_tab).
The otu_tab and a_meta_num I used is attached below in RDS format ('data.zip'). The entry of the otu_tab is defined as raw_counts divided by total_counts_in_the_sample, which is similar to the one presented in your manuscript except that it doesn't multiply by 10^6.
data.zip
Could you please help me look into this? Thanks in advance!
Also, there's a small typo in README. It should be
install.packages("glmnet")
instead of
install.pacakges("glmnet").
Best,
Zifan

Hi Zifan,
Since we are using a gamma normal mixture model to identify the zero counts or low values we will impute on, the smallest number cannot be 0. It should be a small positive number instead. You can use the current data scaled by a scalar (e.g. the median of the total counts in each sample) and take log10(scaled_matrix + 1.01) to get a matrix with smallest number not equal to 0.

I encountered the same error even if I replaced 0 with a small number log10(1.01) or log10(1.1)


Documented structure of Sphere project credential files, both JSON and non-JSON formats

Documented structure of Sphere project credential files, both JSON and non-JSON formats.
Okay, the former is reasonably obvious, but the latter, with its colon separators, was not to me; and it also took me a while to realise that the filenames without the .json extension were not expected to be expressed in JSON format at all.

Coverage remained the same at 24.053% when pulling 4e6fd08cfa4452f8a5c17c05b0aaf5732e720bc1 on feature/improve-readme-documentation into e28d9dea121d73e48542df0fb7e11181b996118f on master.

👍

Coverage remained the same at 24.053% when pulling 9a58fe6af862ed79386b1384381016dec3e4b3b5 on feature/improve-readme-documentation into e28d9dea121d73e48542df0fb7e11181b996118f on master.


Feature request To add ANN model Project

Customer churn prediction is a crucial aspect of business strategy, particularly in the telecom industry, where customer retention is key to maintaining revenue and competitiveness. By building a deep learning model to predict customer churn, telecom operators can identify high-risk customers and take proactive measures to retain them.
Why Customer Churn Prediction is Important
Customer churn can result in significant revenue loss for telecom operators
Identifying high-risk customers allows operators to take targeted retention measures
Deep learning models can accurately predict churn behavior based on customer data
Building a Deep Learning Model for Customer Churn Prediction
Train a deep learning model using the training data, such as a neural network or gradient boosting model
Evaluate the model's performance using metrics such as precision, recall, and F1-score
Evaluating Model Performance.
Precision: measures the proportion of true positives (correctly predicted churners) among all predicted churners
Recall: measures the proportion of true positives among all actual churners
F1-score: measures the harmonic mean of precision and recall, providing a balanced evaluation of the model's performance
Features can be added in this project : data analysis , data cleaning, neural network
@UppuluriKalyani  you liked my idea  then assign me task so that i can work on it !

@UppuluriKalyani  can you please level you assigned!
As it includes one important feature!

@SimranShaikh20 it's ok will add based on your implementation don't worry

Yeah  it's completely fine!

@UppuluriKalyani I had make PR can you review it !

@UppuluriKalyani  can you please change assigned  level  !


Added pre-req link

Added pre-req link for Windows Defender Antivirus.
Problem: https://github.com/MicrosoftDocs/windows-itpro-docs/issues/4030

@greg-lindsay - PR has been copy edited and is ready for final review. This is based on issue #4030.
Thank you.


Mike-Bug: Inadequate error feedback in device context

🐞 Bug Report
Affected Command
.\edgex-cli.exe device add
Add Devices:
Indication that I am missing flags, or in this case a file is not provided/or found should be indicated.
Is this a regression?
No
Description and Minimal Reproduction
🔥 Exception or Error

🌍  Your Environment
Deployment Environment:
EdgeX Fuji
Anything else relevant?

fixed by OR #144


[Draft] Sameerpatro/new api ase jan2023 v1

Choose a PR Template
Switch to "Preview" on this description then select one of the choices below.
Click here to open a PR for a Data Plane API.
Click here to open a PR for a Control Plane (ARM) API.

/azp run


Remove vendor folder. Slightly update structure. Add default Prometheus metrics

Coverage decreased (-0.2%) to 65.067% when pulling 5266064df9fc9753c1c5264bc76568dd55b26dca on add-metrics into 4f1237a9105ec0e029222b82b34f2c1ccfcbef80 on master.


Repos

Refer to this link for build results (access rights to CI server needed):
https://fabric8-ci.fusesource.com/job/wildfly-camel-pullreq/256/
Test FAILed.

retest this please

Refer to this link for build results (access rights to CI server needed):
https://fabric8-ci.fusesource.com/job/wildfly-camel-pullreq/257/
Test FAILed.

retest this please

Refer to this link for build results (access rights to CI server needed):
https://fabric8-ci.fusesource.com/job/wildfly-camel-pullreq/258/
Test PASSed.


Encounter an error when importing the Python API installed by Pip

Describe the bug
Hi there. After installing Shuriken with Pip and importing the Python API, I got the following ImportError.
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/.venv/lib/python3.12/site-packages/shuriken/__init__.py", line 53, in <module>
    raise ImportError("ERROR: fail to load the dynamic library")
ImportError: ERROR: fail to load the dynamic library

It seems like Pip didn't install the required library for Shuriken.
Step to Reproduce

Set up a Python virtual environment on Ubuntu 24.04.python3 -m venv .venv && source .venv/bin/activate

Install Shuriken with Pip.pip install git+https://github.com/Fare9/Shuriken-Analyzer.git@main#subdirectory=shuriken/bindings/Python/

Import the Python API.python3 -c "from shuriken import *"

The ImportError is raised.

Additional Context
By re-running the installation with the -v flag, pip reported a cmake command failed. So, I doubt if there were issues when Pip built Shuriken.
These are part of the Pip output. Starting from line 8, you can see a CMakeError.
$> pip install -v git+https://github.com/Fare9/Shuriken-Analyzer.git@main#subdirectory=shuriken/bindings/Python/

Using pip 24.0 from /.venv/lib/python3.12/site-packages/pip (python 3.12)
Collecting git+https://github.com/Fare9/Shuriken-Analyzer.git@main#subdirectory=shuriken/bindings/Python/
...
  -- Test option used and compilers path provided, compiling DEX files
  -- Path to DEX Folder: /tmp/pip-req-build-drpt5qd4/shuriken/build/tests/code/java/
  CMake Error at CMakeLists.txt:195 (install):                  <------------------------------- Line 8
    install DIRECTORY given no DESTINATION!

  -- Configuring incomplete, errors occurred!
  ERROR:__main__:Cmake build failed: Command '['cmake', '..', '-DCMAKE_BUILD_TYPE=Release']' returned non-zero exit status 1.
  INFO:root:running install_lib
...

hi @haeter525, the issue should be resolved soon in #103 along with a dedicated ci for the python installer so it doesn't happen again on main branch.
The job for proof of installer working is https://github.com/Shuriken-Group/Shuriken-Analyzer/actions/runs/12207652888/job/34059470491


Add MovinetA0-MoViNetA5

添加了MoViNetA0-MoViNetA5的frame形式，修改UCF-101数据集那边的路径即可使用

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Dockerize the hdfs mesos framework to be able to schedule as docker on marathon/chronos

I want to run the hdfs along with the compute for the data locality. It helps me to  better manage everything (hdfs/compute etc) if everything is dockerized from the perspective of the framework execution. Specific nodes be managed as linux container by mesos.

+1

Yes, it would be great to have an example of Dockerfile and related settings..


tests failing

with my repo on 4762f65 I am experiencing one test failure; open to see details

pytest
=========================================================================== test session starts ============================================================================
platform darwin -- Python 3.12.3, pytest-8.1.1, pluggy-1.4.0
rootdir: /private/tmp/IKPLS
configfile: pyproject.toml
collected 26 items
tests/test_ikpls.py ..........F...............                                [100%]
===================================== FAILURES ======================================
___________________________ TestClass.test_gradient_pls_1 ___________________________
self = <tests.test_ikpls.TestClass object at 0x156a850d0>
def test_gradient_pls_1(self):
    """
    Description
    -----------
    This test loads input predictor variables and a target variable with a single
    column and calls the 'check_gradient_pls' method to validate the gradient
    propagation for reverse-mode differentiable JAX PLS.

    Returns:
    None
    """
    X = self.load_X()
    Y = self.load_Y(["Protein"])
    num_components = 25
    filter_size = 7
    assert Y.shape[1] == 1

  self.check_gradient_pls(

        X=X,
        Y=Y,
        num_components=num_components,
        filter_size=filter_size,
        val_atol=0,
        val_rtol=1e-5,
        grad_atol=0,
        grad_rtol=1e-5,
    )

tests/test_ikpls.py:2191:

tests/test_ikpls.py:2173: in check_gradient_pls
assert_allclose(output_val_alg_2, output_val_diff_alg_2, atol=0, rtol=2e-11)

args = (<function assert_allclose..compare at 0x15a5949a0>, array(12.5455451), array(12.5455451))
kwds = {'equal_nan': True, 'err_msg': '', 'header': 'Not equal to tolerance rtol=2e-11, atol=0', 'verbose': True}
@wraps(func)
def inner(*args, **kwds):
    with self._recreate_cm():

      return func(*args, **kwds)

E           AssertionError:
E           Not equal to tolerance rtol=2e-11, atol=0
E
E           Mismatched elements: 1 / 1 (100%)
E           Max absolute difference: 3.54010155e-10
E           Max relative difference: 2.8217997e-11
E            x: array(12.545545)
E            y: array(12.545545)
/Users/tparment/miniconda3/envs/my-ikpls/lib/python3.12/contextlib.py:81: AssertionError
------------------------------- Captured stdout call --------------------------------
stateless_fit for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_preprocess_input_matrices for Improved Kernel PLS Algorithm #1 will be JIT compiled...
get_means for Improved Kernel PLS Algorithm #1 will be JIT compiled...
get_stds for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_get_initial_matrices for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_1 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_main_loop_body for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_2 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_3 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_3_body for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_4 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_5 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
stateless_predict for Improved Kernel PLS Algorithm #1 will be JIT compiled...
stateless_fit for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_preprocess_input_matrices for Improved Kernel PLS Algorithm #2 will be JIT compiled...
get_means for Improved Kernel PLS Algorithm #2 will be JIT compiled...
get_stds for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_get_initial_matrices for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_1 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_main_loop_body for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_2 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_3 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_3_body for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_4 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_5 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
stateless_predict for Improved Kernel PLS Algorithm #2 will be JIT compiled...
stateless_fit for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_preprocess_input_matrices for Improved Kernel PLS Algorithm #1 will be JIT compiled...
get_means for Improved Kernel PLS Algorithm #1 will be JIT compiled...
get_stds for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_get_initial_matrices for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_1 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_main_loop_body for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_2 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_3 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_3_body for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_4 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
_step_5 for Improved Kernel PLS Algorithm #1 will be JIT compiled...
stateless_predict for Improved Kernel PLS Algorithm #1 will be JIT compiled...
stateless_fit for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_preprocess_input_matrices for Improved Kernel PLS Algorithm #2 will be JIT compiled...
get_means for Improved Kernel PLS Algorithm #2 will be JIT compiled...
get_stds for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_get_initial_matrices for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_1 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_main_loop_body for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_2 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_3 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_3_body for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_4 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
_step_5 for Improved Kernel PLS Algorithm #2 will be JIT compiled...
stateless_predict for Improved Kernel PLS Algorithm #2 will be JIT compiled...
============================== short test summary info ==============================
FAILED tests/test_ikpls.py::TestClass::test_gradient_pls_1 - AssertionError:
===================== 1 failed, 25 passed in 1403.61s (0:23:23) =====================

and oops I forgot to link the meta-issue
https://github.com/openjournals/joss-reviews/issues/6533

Hi @parmentelat ,
This appears to be an issue when comparing floats for equality due to the error tolerance being too low. As we have previously discussed, the discrepancy between methods will vary between Python versions and operating system. For the JAX implementations, I suspect they also depend on the hardware (CPU/GPU/TPU).
From your test result I can see that the relative error is about 3^-11 and the absolute error is about 3^-10. Given this rather low value and the fact that the tests pass on my own machine and the 12 different instances in the GitHub action test environment, I think it is safe to assume that the test failing is not due to a bug. As such, I have updated the relative and absolute tolerance for the specific test to be 10^-8. The changes have been committed in https://github.com/Sm00thix/IKPLS/commit/1507921810cb1fa9c073991f1e1c62e707b7eb0f


Health services utilities

Building on https://developer.android.com/training/wearables/health-services
Areas to consider

Service implementation for correct handling of activities
Storage of Health Data and Activity in Room DB
Ongoing Activity template
Mobile/Watch data layer and data syncing.

Tracking elsewhere


Allow environment variables to be specified on pip_requirements resource

I have psycopg2 in my requirements.txt which requires pg_config to be on the PATH. Mine wasn't so the install fails.
It seems like it might be useful to be able to specify environment variables on the pip_requirements resource for stuff like this, unless there's a better way to be going about this.
As a workaround I added a symlink to a location I knew was already on the existing PATH:
link "/usr/local/bin/pg_config" do
	to "/usr/pgsql-#{node["postgresql"]["version"]}/bin/pg_config"
end

I think this one may also help with my broken uwsgi installation via pip_requirements as well. It ended up being because of /opt/chef/embedded/bin being on the PATH.
I was able to work around it by using an execute resource and resetting PATH:
execute "pip install -r requirements.txt" do
	environment({
		"PATH" => "/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin"
	})
end


How does this extension work?

Hi! Just wanted to know how this extension works. As you might be aware, Quizlet has put almost all of its functions behind a paywall... I've just tried the extension out, and after changing the manifest.json to fix some errors, it doesn't work for those situations...
Would be quite useful if you could explain what the extension does, and I'd be more than happy to contribute.

Basically, it creates a new account via the Quizlet API, which cheekily doesn't implement any captcha challenge:
https://github.com/rospino74/Quizlet-Bypass/blob/481a1e834c8b7f3a1ea2b5df5274affb144847d7/src/foreground/quizlet/import/accountMaker.js#L29-L47
And then set the cookies for the page with the newly created session token:
https://github.com/rospino74/Quizlet-Bypass/blob/481a1e834c8b7f3a1ea2b5df5274affb144847d7/src/background/cookieReplacer.ts#L20-L24
At the moment I am also thinking of a way to bypass a possible captcha.


Hs/0.5.0rc2 changes

Using this branch to track changes to be merged to rc2.
This fixes #475

Any reason why our previous lock didn't work properly?

I believe it was silently exploding, but I can't remember why. I wrote tests this time :angel:

Did you re-generate the package-lock from scratch? might be a good idea
Is npm audit coming out clean?

Will check/do

                       === npm audit security report ===

found 0 vulnerabilities
 in 1595 scanned packages


Running into panic when trying to install bin to be self managed

Hi there,
I downloaded bin from the releases page on my System (Arm based) - and tried the install command to let bin manage itself:
   • Getting latest release for marcosnils/bin
   • Starting download of https://api.github.com/repos/marcosnils/bin/releases/assets/164207200
15.89 MiB / 15.89 MiB [---------------------------------------------------------------------------------------------] 100.00% 14.18 MiB p/s 1s
   • Copying for<EMAIL_ADDRESS>into /Users/ista-alexanderpadberg/.local/bin
panic: assignment to entry in nil map

goroutine 1 [running]:
github.com/marcosnils/bin/pkg/config.UpsertBinary(0x14000440150)
        /home/marcos/Projects/bin/pkg/config/config.go:109 +0x3c
github.com/marcosnils/bin/cmd.newInstallCmd.func1(0x14000147900?, {0x140003cd260?, 0x4?, 0x1048f530f?})
        /home/marcos/Projects/bin/cmd/install.go:75 +0x318
github.com/spf13/cobra.(*Command).execute(0x14000152900, {0x140003cd230, 0x1, 0x1})
        /home/marcos/Projects/gopath/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:916 +0x66c
github.com/spf13/cobra.(*Command).ExecuteC(0x14000152600)
        /home/marcos/Projects/gopath/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:1044 +0x320
github.com/spf13/cobra.(*Command).Execute(...)
        /home/marcos/Projects/gopath/pkg/mod/github.com/spf13/cobra@v1.6.1/command.go:968
github.com/marcosnils/bin/cmd.(*rootCmd).Execute(0x1400000d2f0, {0x1400001e1c0, 0x2, 0x140003cd030?})
        /home/marcos/Projects/bin/cmd/root.go:36 +0x144
github.com/marcosnils/bin/cmd.Execute({0x1400006a1c0, 0x6b}, 0x104a24ac0?, {0x1400001e1c0, 0x2, 0x2})
        /home/marcos/Projects/bin/cmd/root.go:26 +0xc0
main.main()
        /home/marcos/Projects/bin/main.go:21 +0xa0

As I am guessing, this has something to do with my config (I have a second system (Linux)) that I share my dot files with:
File: ~/.config/bin/config.json
   {
        "default_path": "$HOME/.local/bin"
   }

Any help is much appreciated, if you need more information, please let me know!

hey! thx for reporting.

Maybe a little more on the procedure I was doing:

Installed and configured bin on my Linux system - everything works here
I synced my configs from my linux system to my macos system (pasted the synced config above)
Now I tried to download the bin binary from the releases page and tried the command bin install github.com/marcosnils/bin

ok, perfect.  This is enough for reproducing.

bin ensure $bin_path doesn't yield any output - but also the $bin_path variable is not set on my macos system

what I meant by this is bin ensure $HOME/.local/bin/bin.

Yes also after installing bin via bin - the binary is placed under ~/.local/bin/bin
and downloading other binaries also work but all off them throw the reported error
bin ensure $HOME/.local/bin
   ⨯ command failed            error=binary path /Users/REDACTED/.local/bin not found

do you have the $HOME/.local/bin  directory created in your new system? AFAIK that's generally not a default dir in MacOS so you'll probably have to create it manually :pray:

Ah yes sorry - this one is also synced/created manually sorry for that - but I tried the installation only afterwards

Unfortunately also doesn't work:
bin ensure $HOME/.local/bin/bin                                                                                 ✔  18:59:20
   ⨯ command failed            error=binary path /Users/REDACTED/.local/bin/bin not found

Unfortunately also doesn't work:

Thx. I assume /Users/REDACTED/.local/bin/bin actually exists, correct?

Yes it exists - Owner + Permission should also be correct - I assume as bin cannot write config it has some conflicts maybe?

Yes it exists - Owner + Permission should also be correct - I assume as bin cannot write config it has some conflicts maybe?

oh, I think I know that the issue is. It's something that I'm currently fixing for the new release. If you check your bin config.json file, you'll notice that the path field for the binaries currently have your linux absolute path (which doesn't exist in mac).
If you replace the absolute paths manually in your config.json to $HOME/.local/bin/$binary_name` that will solve your issue.

Is there maybe another location for the config which might be loaded first? - I already looked at my logical place:
~/.bin/config.json- but nothing

Is there maybe another location for the config which might be loaded first? - I already looked at my logical place:

oh! I thought you copied the config file from other systems where you have some other binaries install. I see the issue now. Could you change the config as follows and try again?
{
    "default_path": "$HOME/.local/bin",
    "bins": {}
}

Yes adding the empty map fixed it :)  - I propably didn't sync after installed other bins - sorry for that - just copied the basic config

Thank you very much!

fixed and released in v0.17.6


Feature Request: Impl AsPrimitive for NonZero

Background
I use async-graphql which depends on this crate on validation.
https://github.com/async-graphql/async-graphql/blob/master/src/validators/minimum.rs
I want to use NonZero Integers as well as normal integers.
Feature Request
impl AsPrimitive for NonZero integers

I use async-graphql which depends on this crate on validation.
https://github.com/async-graphql/async-graphql/blob/master/src/validators/minimum.rs

Given that AsPrimitive is lossy, I think it's quite dangerous to use for validation, even aside from NonZero. For example, minimum(&-1, 032) will return Ok!

Use-case aside -- we aren't currently implementing this for any non-primitive types, as I mentioned in https://github.com/rust-num/num-traits/pull/334#issuecomment-2357065696. That's not necessarily a show-stopper, since the trait docs do mention newtypes too, but it would be an expansion from what we have now.

Thank you for telling me!
I'll close this issue.


Provide more convenient way to update bool fields

Describe the feature
It would be very useful to be able to update bool values (even when they are the zero value false) using the form of Update() in which the struct is passed in. For example:
db.Model(&some_struct).Updates(&some_struct)

A gorm configuration option would be useful that specified that bool values specifically should always be updated regardless of whether it is the zero value or not.
(I know the current behavior is documented in Updates multiple columns. I'm asking for a way to override that behavior for bool fields.)
Motivation
This would be very convenient for 2 reasons:

Unlike the rest of the golang types, with bool values the zero-value (false) is 1 of the "valid" values. So changing a bool field from true to false is a common need.
The struct form of Updates() is very convenient because it enables its use in utility packages in which the code doesn't readily know the exact struct type being updated.

Related Issues

None

Why are bool type special, and in fact the zero value of int when you really need to update it.

If you need to update zero, does Select("*").Update meet your needs?
https://gorm.io/docs/update.html#Update-Selected-Fields
If you really only need to update bool type zero value, you can use BeforeUpdate to change Statement Select/Omit .
You can also register all model hooks by Reflect.
https://gorm.io/docs/update.html#Update-Hooks

func (u *User) BeforeUpdate(tx *gorm.DB) (err error) {
	tx.Statement.Select("*")
	if u.OtherZeroValue == 0 {
		tx.Statement.Omit("OtherZeroValue")
	}
	return
}

It seems that this update zero value field is rarely encountered, but it is very cumbersome every time.
I not sure it needs to be added to gorm, so i create a plugin to support it.
https://github.com/a631807682/zerofield

@a631807682  Thanks for the suggestions! I will try them out.

@a631807682 Just following up: your first suggestion above did exactly what we want. Thx!!

can't believe we talking about this in 21st century


Did the Import feature disappear?

the dropdown next to "Add Record" seems to have disappeared and I have no way to import logs for my vehicles.

Do you mean the import via CSV feature?

Yes

It's still there you just have to enable it in the settings?

Oh JFC I am blind, sorry!

I don't see this feature either.
Searching the docs only seems to reference setting   "EnableCsvImports": true or enabling via the GUI

I don't see where I can actually use this feature from the GUI.
Probably best to add this to documentation.

@jacobburrell
https://docs.lubelogger.com/Records/Service Records#searching-records


Add prohibitDestructiveCommands Method to Prevent Running Destructive Commands in Production

Hi,
In this pull request, I’ve added a prohibitDestructiveCommands method to the Module facade. This method allows you to indicate whether destructive Artisan commands should be prohibited, especially in production environments.
The commands affected are:

module:migrate-fresh
module:migrate-refresh
module:migrate-reset

Usage:
To enable this feature, add the following line to your service provider:
Module::prohibitDestructiveCommands($this->app->isProduction());

This ensures that these potentially harmful commands are disabled when running in a production environment, adding an extra layer of safety for your application.
Additionally, this PR updates the confirmable messages to provide clearer communication to users when running these commands.
Thanks.

awesome, thanks!


wifi now connects to roborange and not ryan's home wifi

API/JSON error 404 note for later

Please resubmit without including the zip file as there is a conflict


Adding tags to quotes according to their mantra

similar to #21
please don't open multiple similar issues.


"Run Test" Seemingly not responding

In Waypoint: SIze Your Image. I believe my code is correct, but the "run test" button is seemingly unresponsive. Code attached below.

Me too


add support to rtl for some language like persian & arabic

if there is possible add a feature so that we simply change layout while the language is Persian or arabic
the menu open from left side so after adding that feature we may open from right side based on language
because the menu in this language should open from right side

Hi, @Mahdimm
We added this feature request to our todo list.
Thanks for using the library.

Any Updates ?!!

Not yet. This feature will not be added in the nearest 2.1 release.
But you always could send us pull-request with rtl implementation if you need it now.
Thank you!

RTL right side open menu option needed !

Any update

This would take some time to complete and test. I can add it to the roadmap of the project but, unfortunately, am not able to give a timeline.
I am beginning to make updates to this project as time allows. Some days there’s more time than others.
Sent with GitHawk

Needed Please !!! RTL

anything yet on this?


This is just a minor enhancement as I noticed it's quite easy to make the garbage collectors job a little easier with quite a minor change in the framework. The code should be quite self explanatory too.

Because of how this class/struct is used by GetForwarder (return type is IObserver rather than a generic constrained by IObserver), this would result in boxing.

Thanks for the hint! I didn't know about the unique behaviour regarding boxing when it comes to structs but I read up on it now. As you pointed out it results in an additional boxing operation which negates the benefit hence why I'll retract the proposed change now.


Doc Update for Feature 1206272

[X] This contribution adheres to CONTRIBUTING.md.

TODO: Check the above box with an 'x' indicating you've read and followed CONTRIBUTING.md.
What does this Pull Request accomplish?
Makes update to getting started documentation to reflect expanded FXP support for feature 1206272.

Hey @zszilagy, I can't seem to add you as a reviewer. Do you want to add yourself?


Steering is broken

Steering is not working properly yet. If there are dark patches left of the lane markings (eg the left lane extends to the right half of the screen) the camera thinks the road extends to the left side of the view and wants to turn there.
See attached photo for clarification

This problem was solved


Is the model available?

Hi, is your pretrained model available? I am researching pretrained models for my schema matching task. Thank you

Hi, is your pretrained model available? I am researching pretrained models for my schema matching task. Thank you

It seems the model URL are in urls_pretrained_model.py file


Removing problematic submodule and adding them as pre-built binaries.

We may want to go back to this someday, but for now we'll just use the pre-built libraries.

merged


[SPARK-4879] Use driver to coordinate Hadoop output committing for speculative tasks

Previously, SparkHadoopWriter always committed its tasks without question. The problem is that when speculation is enabled sometimes this can result in multiple tasks committing their output to the same file. Even though an HDFS-writing task may be re-launched due to speculation, the original task is not killed and may eventually commit as well.
This can cause strange race conditions where multiple tasks that commit interfere with each other, with the result being that some partition files are actually lost entirely. For more context on these kinds of scenarios, see SPARK-4879.
In Hadoop MapReduce jobs, the application master is a central coordinator that authorizes whether or not any given task can commit. Before a task commits its output, it queries the application master as to whether or not such a commit is safe, and the application master does bookkeeping as tasks are requesting commits. Duplicate tasks that would write to files that were already written to from other tasks are prohibited from committing.
This patch emulates that functionality - the crucial missing component was a central arbitrator, which is now a module called the OutputCommitCoordinator. The coordinator lives on the driver and the executors can obtain a reference to this actor and request its permission to commit. As tasks commit and are reported as completed successfully or unsuccessfully by the DAGScheduler, the commit coordinator is informed of the task completion events as well to update its internal state.
Future work includes more rigorous unit testing and extra optimizations should this patch cause a performance regression. It is unclear what the overall cost of communicating back to the driver on every hadoop-committing task will be. It's also important for those hitting this issue to backport this onto previous version of Spark because the bug has serious consequences, that is, data is lost.
Currently, the OutputCommitCoordinator is only used when spark.speculation is true.  It can be disabled by setting spark.hadoop.outputCommitCoordination.enabled=false in SparkConf.
This patch is an updated version of #4155 (by @mccheah), which in turn was an updated version of this PR.
Closes #4155.

Test build #25629 has started for   PR 4066 at commit c25c997.

This patch merges cleanly.

Test build #25629 timed out     for PR 4066 at commit c25c997     after a configured wait of 120m.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25629/
Test FAILed.

Test build #25640 has started for   PR 4066 at commit 8c64d12.

This patch merges cleanly.

Test build #25640 has finished for   PR 4066 at commit 8c64d12.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class AskPermissionToCommitOutput(

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25640/
Test FAILed.

A large fraction of the test failures in DAGSchedulerSuite are because it passes null for CompletionEvent.taskInfo.  I don't think that this field is actually nullable in real Spark jobs, so we should probably fix up those tests to include a mock TaskInfo objects in the events that they construct.

Test build #25679 has started for   PR 4066 at commit 63a7707.

This patch merges cleanly.

Test build #25679 timed out     for PR 4066 at commit 63a7707     after a configured wait of 120m.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25679/
Test FAILed.

Instead of having every task require a call back to the driver or master, can the master broadcast to the executor that a task is being speculated and any executor with a copy of that task should do a commit verification? Just throwing the idea out there.

@mccheah I agree that we could add optimizations to skip this coordination in scenarios where we know that there will only be one copy of a task, but it might be a good idea to defer those optimizations to a separate patch if it turns out that the performance impact is small.  Once we've got a version of this patch that passes tests, let's benchmark it and see how costly this extra coordination is.

Jenkins, retest this please.

(Just so we can see the test failure message again)

Test build #25834 has started for   PR 4066 at commit 63a7707.

This patch does not merge cleanly.

Did you think of any corner cases that you might have missed? In terms of correctness, this seems okay (although the Jenkins build indicates there are some issues). Have you thought about testing? I'm trying to brainstorm how to unit test this as well.

Hmm, let me think about this a bit...
I don't think that we have any end-to-end tests with task speculation right now, since Spark won't schedule speculated tasks on executors that are running on the same host.  However, I don't think that end-to-end tests should be the only/best way to test this, since there are a few special cases that require us to have finer-grained control over the event interleaving.
Here are some of the scenarios that I'd like to test:

Two copies of a task are running and both try to commit at (essentially) the same time.  In other words, needsTaskCommit should return true for both tasks, but only one should be allowed to commit.
The task that is authorized to commit crashes before committing.  In this case, a new copy of the task should be scheduled and that task should be able to successfully commit its output (i.e. the "lock" held by the winning task should be released if that task dies without completing the commit process).

Since this involves an interplay between DAGScheduler logic, OutputCommitCoordinator, and actual tasks, we'll might have to use a mock TaskSchedulerBackend and mock task to test this.

Test build #25834 has finished for   PR 4066 at commit 63a7707.

This patch fails Spark unit tests.
This patch does not merge cleanly.
This patch adds the following public classes (experimental):
case class AskPermissionToCommitOutput(

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25834/
Test FAILed.

The linked pull request takes your ideas, makes them compatible with master, and adds unit tests. Feel free to take a look.

@mccheah Thanks for picking this up.  I'm going to close my PR and continue discussion on yours.

Test build #26629 has started for   PR 4066 at commit 63a7707.

This patch does not merge cleanly.

I've pulled @mccheah's commits into my PR and am going to work on trying to finish this for 1.3.0.

Test build #26630 has started for   PR 4066 at commit 594e41a.

This patch merges cleanly.

Test build #26637 has started for   PR 4066 at commit 92e6dc9.

This patch merges cleanly.

Test build #26629 has finished for   PR 4066 at commit 63a7707.

This patch fails Spark unit tests.
This patch does not merge cleanly.
This patch adds the following public classes (experimental):
case class AskPermissionToCommitOutput(

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26629/
Test FAILed.

Test build #26645 has started for   PR 4066 at commit c79df98.

This patch merges cleanly.

This patch looks like it's in pretty good shape.  I did a bit of code cleanup in OutputCommitCoordinator to reduce the visibility of some methods, clarify when / where certain methods are invoked, and to more clearly explain the API contract of canCommit.  I also fixed a bug which I believe was causing the CheckpointSuite failures that we observed on @mccheah's PR.
I've still got to take a closer look at the unit tests and maybe add a new test to catch the bug that I fixed.  I also have to manually test this on a cluster to see if it fixes the reproduction of the original bug that prompted this patch.  I'll try to get to this tomorrow so this can make it into 1.3.0.

Test build #26630 timed out     for PR 4066 at commit 594e41a     after a configured wait of 120m.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26630/
Test FAILed.

Test build #26637 timed out     for PR 4066 at commit 92e6dc9     after a configured wait of 120m.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26637/
Test FAILed.

Test build #26645 timed out     for PR 4066 at commit c79df98     after a configured wait of 120m.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26645/
Test FAILed.

Approach looks ok to me, as long as needTaskCommit isn't broken in some way, and could tell a task that it shouldn't commit its output when in fact it should.

Test build #26677 has started for   PR 4066 at commit dd00b7c.

This patch merges cleanly.

I was able to reproduce one of the test failures locally:
[info] - repeatedly failing task that crashes JVM *** FAILED *** (1 minute, 40 seconds)
[info]   The code passed to failAfter did not complete within 100000 milliseconds. (DistributedSuite.scala:123)
[info]   org.scalatest.exceptions.TestFailedDueToTimeoutException:
[info]   at org.scalatest.concurrent.Timeouts$$anonfun$failAfter$1.apply(Timeouts.scala:249)
[info]   at org.scalatest.concurrent.Timeouts$$anonfun$failAfter$1.apply(Timeouts.scala:249)
[info]   at org.scalatest.concurrent.Timeouts$class.timeoutAfter(Timeouts.scala:345)
[info]   at org.scalatest.concurrent.Timeouts$class.failAfter(Timeouts.scala:245)
[info]   at org.scalatest.concurrent.Timeouts$.failAfter(Timeouts.scala:358)
[info]   at org.apache.spark.DistributedSuite$$anonfun$11.apply$mcV$sp(DistributedSuite.scala:123)
[info]   at org.apache.spark.DistributedSuite$$anonfun$11.apply(DistributedSuite.scala:117)
[info]   at org.apache.spark.DistributedSuite$$anonfun$11.apply(DistributedSuite.scala:117)
[info]   at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)
[info]   at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
[info]   at org.scalatest.Suite$class.withFixture(Suite.scala:1122)
[info]   at org.scalatest.FunSuite.withFixture(FunSuite.scala:1555)
[info]   at org.scalatest.FunSuiteLike$class.invokeWithFixture$1(FunSuiteLike.scala:163)
[info]   at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
[info]   at org.scalatest.FunSuiteLike$$anonfun$runTest$1.apply(FunSuiteLike.scala:175)
[info]   at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306)
[info]   at org.scalatest.FunSuiteLike$class.runTest(FunSuiteLike.scala:175)
[info]   at org.apache.spark.DistributedSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(DistributedSuite.scala:31)
[info]   at org.scalatest.BeforeAndAfterEach$class.runTest(BeforeAndAfterEach.scala:255)
[info]   at org.apache.spark.DistributedSuite.runTest(DistributedSuite.scala:31)
[...]

I'm not sure what's causing this, since I don't think this test even attempts to save Hadoop output.  Maybe OutputCommitCoordinator is throwing an exception and crashing the DAGScheduler?

Test build #26682 has started for   PR 4066 at commit 459310a.

This patch merges cleanly.

Test build #26686 has started for   PR 4066 at commit 997b41b.

This patch merges cleanly.

Test build #26691 has started for   PR 4066 at commit f582574.

This patch merges cleanly.

Test build #26682 has finished for   PR 4066 at commit 459310a.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)
class GaussianMixtureModel(object):
class GaussianMixture(object):
class MultivariateGaussian(namedtuple('MultivariateGaussian', ['mu', 'sigma'])):
class KafkaUtils(object):
public class JDBCUtils
trait Column extends DataFrame with ExpressionApi
class ColumnName(name: String) extends IncomputableColumn(name)
trait DataFrame extends DataFrameSpecificApi with RDDApi[Row]
class GroupedDataFrame protected[sql](df: DataFrameImpl, groupingExprs: Seq[Expression])
 protected[sql] class QueryExecution(val logical: LogicalPlan)
          logWarning(s"Couldn't find class $driver", e);
 implicit class JDBCDataFrame(rdd: DataFrame)

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26682/
Test PASSed.

There's one subtle, extremely-unlikely race condition that can still occur here and which I would like to fix: currently, we seem to assume that messages sent from the DAGScheduler to the local OutputCommitCoordinator will be processed before commit requests for tasks.  However, we send these as fire-and-forget messages and do not wait for acknowledgements, so it would technically be legal for OutputCommitCoordinator to receive a request to commit for a task that belongs to a stage that OutputCommitCoordinator has not heard about.  This will cause the coordinator to deny the task attempt, since it will think that it's from a completed stage.  I think that this isn't a huge deal in practice, since I think we'll just end up scheduling another task (plus this particular race should be extremely unlikely).
Still, though, I'd like to consider whether there's a good way to fix this in order to guarantee that requests to commit output for a stage occur after the stage start event (from the OutputCommitter's POV).

Test build #26677 timed out     for PR 4066 at commit dd00b7c     after a configured wait of 120m.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26677/
Test FAILed.

Also, this could probably benefit from a more comprehensive unit test of the commit coordinator itself, since I think it's possible to alter the current implementation in ways that will introduce bugs that won't be caught by the current test (e.g. by swapping the id types).

Test build #26686 has finished for   PR 4066 at commit 997b41b.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26686/
Test PASSed.

Still, though, I'd like to consider whether there's a good way to fix this in order to guarantee that requests to commit output for a stage occur after the stage start event (from the OutputCommitter's POV).

I handed this by refactoring the code so that only remote RPCs are routed through the actor.  The methods called by DAGScheduler are now synchronized and directly update the hashmaps.

Test build #26695 has started for   PR 4066 at commit 97da5fe.

This patch merges cleanly.

Test build #26691 has finished for   PR 4066 at commit f582574.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26691/
Test PASSed.

Test build #26700 has started for   PR 4066 at commit 97da5fe.

This patch merges cleanly.

Test build #26695 has finished for   PR 4066 at commit 97da5fe.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26695/
Test PASSed.

Test build #26700 has finished for   PR 4066 at commit 97da5fe.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26700/
Test PASSed.

After some more thought and testing, I don't know if it's safe to ignore task failures that are due to commits being denied, since doing so risks infinite rescheduling if all commits are denied.  On the other hand, treating these as failures could lead to spurious job failures in cases where you have many copies of one slow, speculated task (the old behavior would treat these as successful task completions).

One approach would be to have non-authorized committers block in the canCommit call until the master knows whether the authorized commit has succeeded.  In these cases, we'd only send commit denied once at least one copy of the task had successfully committed its output.
Another hackier approach would be to not count speculative tasks' failures towards the total task failure count and assume that we'll eventually rack up enough non-speculative failures to abort the job.
Pragmatically, though, we could just choose to ignore this issue: if the implementation is correct, then we should only get into an infinite commit denial loop if the authorized committer keeps failing, which will trigger a job failure because commit failures are counted as task failures.

Test build #26712 has started for   PR 4066 at commit 3969f5f.

This patch merges cleanly.

Test build #26712 has finished for   PR 4066 at commit 3969f5f.

This patch passes all tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26712/
Test PASSed.

I may be missing something, but how are the tests passing given your latest observations and test...?
That being said, with the infinite commit denied problem - each job that attempts to commit and fails will increment the task-failed count by 1 anyways, and eventually the task should give up and no longer be launched.

I think your latest comment is correct, @JoshRosen . We shouldn't hit an infinite loop because the failing authorized committers will eventually cause the task set to give up.

Test build #26852 has started for   PR 4066 at commit 48d5c1c.

This patch merges cleanly.

Test build #26852 has finished for   PR 4066 at commit 48d5c1c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26852/
Test FAILed.

Jenkins, retest this please.

Test build #26865 has started for   PR 4066 at commit 48d5c1c.

This patch merges cleanly.

Test build #26865 has finished for   PR 4066 at commit 48d5c1c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26865/
Test FAILed.

Jenkins, retest this please.

Test build #26874 has started for   PR 4066 at commit 48d5c1c.

This patch merges cleanly.

Test build #26874 has finished for   PR 4066 at commit 48d5c1c.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26874/
Test FAILed.

@zsxwing, given your interest in thread-safety and synchronization issues, you might be interested in this patch.

@JoshRosen Sorry that I missed the synchronized calling in the methods. I deleted the previous comments. LGTM.

I'm going to try to pick this up now to finish a few remaining changes, mostly variable-renaming to avoid re-introducing the confusing "splits" terminology.
I want to quickly draw attention to a comment up-thread, though: https://github.com/apache/spark/pull/4066#discussion_r24315166.  In a nutshell, we might want to enable this even if speculation is turned off because it's possible for concurrent copies of a task to occur if fetch failures trigger re-execution of earlier stages.  This might be a rarer occurrence than speculation races, though.
I also still have to test this on EC2 and need to run some performance benchmarks.

Test build #27221 has started for   PR 4066 at commit 14861ea.

This patch merges cleanly.

Test build #27221 has finished for   PR 4066 at commit 14861ea.

This patch fails to build.
This patch merges cleanly.
This patch adds the following public classes (experimental):
case class TaskCommitDenied(
class CommitDeniedException(
  class OutputCommitCoordinatorActor(outputCommitCoordinator: OutputCommitCoordinator)

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27221/
Test FAILed.

Jenkins, retest this please.

Test build #27223 has started for   PR 4066 at commit 14861ea.

This patch merges cleanly.

Test build #27223 has finished for   PR 4066 at commit 14861ea.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27223/
Test PASSed.

@mccheah Did you get a chance to test this out on your cluster to see whether it fixes your reproduction of the original bug?

(Cross-posted from JIRA)
This issue is really hard to reproduce, but I managed to trigger the original bug as part of the testing for my patch. Here's what I ran:
~/spark-1.3.0-SNAPSHOT-bin-1.0.4/bin/spark-shell --conf spark.speculation.multiplier=1 --conf spark.speculation.quantile=0.01 --conf spark.speculation=true --conf  spark.hadoop.outputCommitCoordination.enabled=false

val numTasks = 100
val numTrials = 100
val outputPath = "/output-committer-bug-"
val sleepDuration = 1000

for (trial <- 0 to (numTrials - 1)) {
  val outputLocation = outputPath + trial
  sc.parallelize(1 to numTasks, numTasks).mapPartitionsWithContext { case (ctx, iter) =>
    if (ctx.partitionId % 5 == 0) {
      if (ctx.attemptNumber == 0) {  // If this is the first attempt, run slow
       Thread.sleep(sleepDuration)
      }
    }
    iter
  }.map(identity).saveAsTextFile(outputLocation)
  Thread.sleep(sleepDuration * 2)
  println("TESTING OUTPUT OF TRIAL " + trial)
  val savedData = sc.textFile(outputLocation).map(_.toInt).collect()
  if (savedData.toSet != (1 to numTasks).toSet) {
    println("MISSING: " + ((1 to numTasks).toSet -- savedData.toSet))
    assert(false)
  }
  println("-" * 80)
}

It took 22 runs until I actually observed missing output partitions (several of the earlier runs threw spurious exceptions and didn't have missing outputs):
[...]
15/02/10 22:17:21 INFO scheduler.DAGScheduler: Job 66 finished: saveAsTextFile at <console>:39, took 2.479592 s
15/02/10 22:17:21 WARN scheduler.TaskSetManager: Lost task 75.0 in stage 66.0 (TID 6861, ip-172-31-1-124.us-west-2.compute.internal): java.io.IOException: Failed to save output of task: attempt_201502102217_0066_m_000075_6861
        at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:160)
        at org.apache.hadoop.mapred.FileOutputCommitter.moveTaskOutputs(FileOutputCommitter.java:172)
        at org.apache.hadoop.mapred.FileOutputCommitter.commitTask(FileOutputCommitter.java:132)
        at org.apache.spark.SparkHadoopWriter.performCommit$1(SparkHadoopWriter.scala:113)
        at org.apache.spark.SparkHadoopWriter.commit(SparkHadoopWriter.scala:150)
        at org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1082)
        at org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1059)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
        at org.apache.spark.scheduler.Task.run(Task.scala:64)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:197)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

15/02/10 22:17:21 WARN scheduler.TaskSetManager: Lost task 80.0 in stage 66.0 (TID 6866, ip-172-31-11-151.us-west-2.compute.internal): java.io.IOException: The temporary job-output directory hdfs://ec2-54-213-142-80.us-west-2.compute.amazonaws.com:9000/output-committer-bug-22/_temporary doesn't exist!
        at org.apache.hadoop.mapred.FileOutputCommitter.getWorkPath(FileOutputCommitter.java:250)
        at org.apache.hadoop.mapred.FileOutputFormat.getTaskOutputPath(FileOutputFormat.java:244)
        at org.apache.hadoop.mapred.TextOutputFormat.getRecordWriter(TextOutputFormat.java:116)
        at org.apache.spark.SparkHadoopWriter.open(SparkHadoopWriter.scala:91)
        at org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1068)
        at org.apache.spark.rdd.PairRDDFunctions$$anonfun$13.apply(PairRDDFunctions.scala:1059)
        at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:61)
        at org.apache.spark.scheduler.Task.run(Task.scala:64)
        at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:197)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

15/02/10 22:17:21 INFO scheduler.TaskSetManager: Lost task 85.0 in stage 66.0 (TID 6871) on executor ip-172-31-1-124.us-west-2.compute.internal: java.io.IOException (The temporary job-output directory hdfs://ec2-54-213-142-80.us-west-2.compute.amazonaws.com:9000/output-committer-bug-22/_temporary doesn't exist!) [duplicate 1]
15/02/10 22:17:21 INFO scheduler.TaskSetManager: Lost task 90.0 in stage 66.0 (TID 6876) on executor ip-172-31-1-124.us-west-2.compute.internal: java.io.IOException (The temporary job-output directory hdfs://ec2-54-213-142-80.us-west-2.compute.amazonaws.com:9000/output-committer-bug-22/_temporary doesn't exist!) [duplicate 2]
15/02/10 22:17:21 INFO scheduler.TaskSetManager: Lost task 95.0 in stage 66.0 (TID 6881) on executor ip-172-31-11-151.us-west-2.compute.internal: java.io.IOException (The temporary job-output directory hdfs://ec2-54-213-142-80.us-west-2.compute.amazonaws.com:9000/output-committer-bug-22/_temporary doesn't exist!) [duplicate 3]
15/02/10 22:17:21 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 66.0, whose tasks have all completed, from pool
TESTING OUTPUT OF TRIAL 22
[...]
15/02/10 22:17:23 INFO scheduler.TaskSchedulerImpl: Removed TaskSet 67.0, whose tasks have all completed, from pool
15/02/10 22:17:23 INFO scheduler.DAGScheduler: Stage 67 (collect at <console>:42) finished in 0.158 s
15/02/10 22:17:23 INFO scheduler.DAGScheduler: Job 67 finished: collect at <console>:42, took 0.165580 s
MISSING: Set(76)
java.lang.AssertionError: assertion failed
[...]

And I confirmed that it's missing in HDFS:
~/ephemeral-hdfs/bin/hadoop fs -ls /output-committer-bug-22 | grep part | wc -l
Warning: $HADOOP_HOME is deprecated.

99

To test my patch, I'm going to remove the flag that disables it and run this for a huge number of trials to ensure that we don't hit the missing output bug.

After over 400 runs, I haven't seen any missing output so I think it's safe to say that this fixed the bug.  I did notice that speculative tasks sometimes fail with exceptions after the job finishes, but they fail in the SparkHadoopWriter.open() call because the _temporary directory has already been deleted.  This is some annoying log noise, but it shouldn't impact correctness.  These spurious task failures won't cause the job to fail because they only occur after the job has committed.

Test build #27254 has started for   PR 4066 at commit 658116b.

This patch merges cleanly.

LGTM, I will merge this once tests pass

Test build #27254 has finished for   PR 4066 at commit 658116b.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27254/
Test FAILed.

retest this please

actually I'm just going to merge this since it's failing streaming tests. Thanks @JoshRosen, this is going into master and 1.3.

Test build #27264 has started for   PR 4066 at commit 658116b.

This patch merges cleanly.

Test build #27264 has finished for   PR 4066 at commit 658116b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/27264/
Test PASSed.

I only merged this into master and 1.3 for now because there are significant merge conflicts. Thanks @JoshRosen and @mccheah.

SPARK-4879  also happened when use saveAsNewAPIHadoopFile.
Why does not support the saveAsNewAPIHadoopFile function?


Show Workflow Instances list scheduled workflow instance error

Describe the bug
When I click on the "Show Workflow Instances" list I get the following error when I try to open a Scheduled workflow instance.
Object reference not set to an instance of an object.
at DevExpress.ExpressApp.Workflow.Visualization.TrackingVisualizationController.OnDeactivated()
at DevExpress.ExpressApp.Controller.activeList_ResultValueChanged(Object sender, BoolValueChangedEventArgs e)
at DevExpress.ExpressApp.Utils.BoolList.OnResultValueChanged(Boolean oldResultValue, Boolean resultValue)
at DevExpress.ExpressApp.Utils.BoolList.EndUpdate()
at DevExpress.ExpressApp.Utils.BoolList.SetItemValue(String key, Boolean value)
at DevExpress.ExpressApp.Frame.<>c__DisplayClass52_0.b__1()
at DevExpress.Persistent.Base.SafeExecutor.Execute(Method method, Object targetObject, String targetObjectIdentifier)
at DevExpress.ExpressApp.Frame.Dispose()
at DevExpress.ExpressApp.Win.WinShowViewStrategyBase.CreateWindow(ShowViewParameters parameters, ShowViewSource showViewSource, Boolean isMain)
at DevExpress.ExpressApp.Win.WinShowViewStrategyBase.CreateInspectorWindow(ShowViewParameters parameters, ShowViewSource showViewSource)
at DevExpress.ExpressApp.Win.MdiShowViewStrategy.CreateInspectorWindow(ShowViewParameters parameters, ShowViewSource showViewSource)
at DevExpress.ExpressApp.Win.MdiShowViewStrategy.ShowViewInInspector(ShowViewParameters parameters, ShowViewSource showViewSource)
at DevExpress.ExpressApp.Win.WinShowViewStrategyBase.ShowViewInNewWindow(ShowViewParameters parameters, ShowViewSource showViewSource)
at DevExpress.ExpressApp.Win.MdiShowViewStrategy.ShowViewFromCommonView(ShowViewParameters parameters, ShowViewSource showViewSource)
at DevExpress.ExpressApp.ShowViewStrategyBase.ShowViewCore(ShowViewParameters parameters, ShowViewSource showViewSource)
at DevExpress.ExpressApp.Win.MdiShowViewStrategy.ShowViewCore(ShowViewParameters parameters, ShowViewSource showViewSource)
at DevExpress.ExpressApp.ShowViewStrategyBase.ShowView(ShowViewParameters parameters, ShowViewSource showViewSource)
at DevExpress.ExpressApp.Frame.ProcessActionCreatedView(ActionBaseEventArgs e)
at DevExpress.ExpressApp.Win.WinWindow.ProcessActionCreatedView(ActionBaseEventArgs e)
at DevExpress.ExpressApp.Frame.ShowViewAction_ProcessCreatedView(Object sender, ActionBaseEventArgs e)
at DevExpress.ExpressApp.Actions.ActionBase.OnProcessCreatedView(ActionBaseEventArgs e)
at DevExpress.ExpressApp.Actions.ActionBase.ExecuteCore(Delegate handler, ActionBaseEventArgs eventArgs)
To Reproduce
Steps to reproduce the behavior:

Go to 'Show Workflow Instances'
Click on a scheduled workflow instance in the list.
See error

Expected behavior
Show the scheduled workflow instance details like the DevExpress workflow details.

thanks for reporting, where exactly did you got this? empty solution? your solution?

My solution
From: Apostolis Bekiaris<EMAIL_ADDRESS>Sent: Wednesday, April 10, 2019 9:07 AM
To: eXpandFramework/eXpand<EMAIL_ADDRESS>Cc: Pratt, Niel W<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [eXpandFramework/eXpand] Show Workflow Instances list scheduled workflow instance error (#365)
Sent by an external sender

thanks for reporting, where exactly did you got this? empty solution? your solution?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/eXpandFramework/eXpand/issues/365#issuecomment-481682873, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AvKPAVUdLmXZv3PN4yQee7AnFMurv-elks5vfeIDgaJpZM4cm7h_.

I think the problem is with the DevExpress.ExpressApp.Workflow.Visualization.RunningWorkflowInstanceInfoControllerBase class trying to parse the WorkflowUniqueId = “ScheduledWorkflowC0D4E34F_A93B_4B2F_B025_F8E5C6E4EDE1_2” in the PatchVersionedXaml method.
I tried to use the workflow demo application, but I could not get a schedule workflow to run.
From: Apostolis Bekiaris<EMAIL_ADDRESS>Sent: Wednesday, April 10, 2019 9:07 AM
To: eXpandFramework/eXpand<EMAIL_ADDRESS>Cc: Pratt, Niel W<EMAIL_ADDRESS>Author<EMAIL_ADDRESS>Subject: Re: [eXpandFramework/eXpand] Show Workflow Instances list scheduled workflow instance error (#365)
Sent by an external sender

thanks for reporting, where exactly did you got this? empty solution? your solution?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/eXpandFramework/eXpand/issues/365#issuecomment-481682873, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AvKPAVUdLmXZv3PN4yQee7AnFMurv-elks5vfeIDgaJpZM4cm7h_.

posting a sample with instructions can speed up things as I am overloaded at the moment.

SystemMonitor.zip
Here is a sample project. Thanks

Thanks for the sample and sorry for the delay. You have to use special classes with the XpandWorkFlow. Look how WorkflowTester solution is initializing the WorkFlowServer
https://github.com/eXpandFramework/eXpand/blob/84e8e3e037bae7a4a1460c6657d7bb40a3fb49ca/Demos/Modules/Workflow/WorkflowTester.Win/Program.cs#L60-L68
in addition there is the Scheduled Workflows easytest that passed ok in my tests.
https://github.com/eXpandFramework/eXpand/blob/84e8e3e037bae7a4a1460c6657d7bb40a3fb49ca/Demos/Modules/Workflow/WorkflowTester.Module/FunctionalTests/ScheduledWorkflow/ScheduledWorkflow.ets#L1-L31

Closing issue for age. Feel free to reopen it at any time.
.Thank you for your contribution.


Sanity checks of a request include "do we have one?".

See HttpRequest::Done for more "information".

lgtm


Inaccurate Earnings Dates

Bug Report
Unfortunately, Earnings Dates from YF are full of inacurate and misleading data.
Describe the bug
Example of approximate earnings dates(a.k.a. unconfirmed) from quote call for Intel($INTC)
  earningsTimestamp: 2021-10-21T20:00:00.000Z,
  earningsTimestampStart: 2022-04-20T20:00:00.000Z,
  earningsTimestampEnd: 2022-04-25T20:00:00.000Z,

As I can see, YF just generates some date range based on previous date. You can see that earningsTimestampStart != earningsTimestampEnd.
The problem is that real earnings event was hold yesterday, Jan 26.
Example of confirmed earnings date from quote call for PayPal($PYPL)
  earningsTimestamp: 2022-02-02T00:00:00.000Z,
  earningsTimestampStart: 2022-02-02T00:00:00.000Z,
  earningsTimestampEnd: 2022-02-02T00:00:00.000Z,

As shown on official website, PayPal’s Q4 2021 Earnings Call is expected on FEBRUARY 1, 2022 02:00 PM PT.
So even for "confirmed" date YF provides wrong date. As you can see, time is full of nulls and I found that if time contain some value, then date is correct when earningsTimestampStart == earningsTimestampEnd. Time in date is for information purposes only and usually wrong, but I found that for many cases if time is <= opening bell time, then earnings will be unveiled BEFORE market open.
I think, docs should warn about that Earnings Dates are full of inacurate and misleading data and better to use another source.

Thanks @glintik, this is very helpful to know.  I agree and have added a note about this in the docs, with a link back to this issue.  Thanks for reporting!

:tada: This issue has been resolved in version 2.1.3 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Add images to exaple presentation

We seem to have major issues with images, should add some...

blocked by #2

done


Don't require an explicit namespace label to exclude webhooks

The v0.8 implementation of our webhooks was for an 'excluded-namespace' label, which means we need to explicitly label namespaces (like kube-system) that we don't want to be affected by HNC. However, a better solution might be to use the list of excluded namespaces on the command-line as the only source of truth, and have HNC automatically add a label to all namespaces that it does manage, and then have the webhooks look for that label instead.
This would remove the need for users to take any explicit steps to exclude critical namespaces. It also gives us a nice path to ensure the object webhook doesn't operate in externally-managed namespaces.
/milestone v0.9
/assign @yiqigao217

Closing this to consolidate with #9


model source will not open?

Unfortunately, we can't publish proprietary training code due to NDA.


Add example of how configure Identity to use Guid data type for primary keys

(Next #1975 PR)
Example of how change the data type of the primary keys with Identity.
It's a need we often have, the is a lot of Stack Overflow posts on the subject.

Hi @AdrienTorris, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;

Note: I didn't know how to fill the md headers (manager, assetid, authors, ...) so I just filled the title and the uid. Other fields are not set.


Make FeatureExtractor accept Numpy Arrays

Resolve #165
Make FeatureExtractor accept Numpy Arrays as input.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


List grouping by spfx computed column does not work - always group as not set

What type of issue is this?
Question
What SharePoint development model, framework, SDK or API is this about?
💥 SharePoint Framework
Target SharePoint environment
SharePoint Online
What browser(s) / client(s) have you tested

[ ] 💥 Internet Explorer
[ ] 💥 Microsoft Edge
[X] 💥 Google Chrome
[ ] 💥 FireFox
[ ] 💥 Safari
[ ] mobile (iOS/iPadOS)
[ ] mobile (Android)
[ ] not applicable
[ ] other (enter in the "Additional environment details" area below)

Additional environment details

browser version
SPFx version
Node.js version
etc

Issue description
Having a custom spfx computed column in a list (wich displays true or false depending if two other fields are equal), when we select "Group by this column", the only group that is displayed is "Not set"  (Non appliqué in french)

Hi @CarlVerret, could you provide information about the calculated column and view grouping setting? I have checked it on my side and I can see the correct grouping label for a calculated column.

Hi mkm17,
In fact I had to use a custom spfx field to compare if, under some circumstances two dates were equal or not, returning a simple Yes or No text value.
public onRenderCell(event: IFieldCustomizerCellEventParameters): void {
const monItemEnCours = event.listItem;

if (  ... ) //   ... under some condition, set value to blank
{
event.domElement.innerText = "";
}
else
{
  const date_approb:Date =  monItemEnCours.getValueByName("approved date");
  const date_modif:Date = monItemEnCours.getValueByName("Modified");
  event.domElement.innerText =  date_modif <=date_approb  ? "Yes" : "No";

}

}
The display works perfectly in list's columns.  But grouping by this column result in only "Non affecté" (not set in french) values.  Even if the cell render produces values.

@CarlVerret Unfortunately, it will not work like this. The SPFx field customizer does not change the value of a field; it just overrides the rendering of the column field. Maybe you can try using a calculated column formula instead. Try comparing both dates with conditional formulas:
https://learn.microsoft.com/en-us/previous-versions/office/developer/sharepoint-2010/bb862071(v=office.14)#conditional-formulas
If there are additional conditions, such as comparing both dates when the status of the items is 'Approved,' then simply add a nested IF statement.

The main reason we opted for a custom field is that there's no easy way to compare two dates natively with calculated columns.  If there's one, we could figure it out !  This is something that is asked a lot about in many forums.


Noskip testing. fixes #12

Running python setup.py test_noskip will now cause an error if a test is skipped. This is implemented in the Dockerfile so that guthub actions require all tests to pass.
python setup.py test is unaffected.

I don't understand why none of the Actions are working (also on other branches).

@tankefugl I don't understand why none of the Actions are working (also on other branches).

Seems they are working again now. maybe something was wrong on the github end?


Update CODEOWNERS for Bengali branch

Describe your changes
We need to update the code owners for the Bengali Branch
Related issue number or link (ex: resolves #issue-number)
Checklist before opening this PR (put x in the checkboxes)

[x] This PR does not contain plagiarism

don’t copy other people’s work unless you are quoting and contributing it to them.

[x] I have signed off on all commits

signing off (ex: git commit -s) is to affirm that commits comply DCO. If you are working locally, you could add an alias to your gitconfig by running git config --global alias.ci "commit -s".

Hi @mitul3737,
We believe sufficient time has passed for a lazy consensus. We would like to replace @Mouly22 and @ikramulkayes with @Arindam200 and @sajibAdhi.
Maintainers will update CODEOWNERS file first (via #2134).
In the meantime, please leave comment here to confirm that each approver candidate (@Arindam200 and @sajibAdhi) understands the policy and information for localization approvers.

#723

Hello, @seokho-son
I understood policies and information for localization approvers.

Hello, @seokho-son!
I fully understand the policies and information for localization approvers.
Thank you.

Thanks! let me close this PR and continue settings in other PRs.

https://github.com/cncf/people/pull/248
https://github.com/cncf/glossary/pull/2134

By the way, I'd like to clarify that we don't say a localization approver is a maintainer of cloud native glossary. :)
https://glossary.cncf.io/contributor-ladder/
When introducing oneself, it would be more accurate to mention a localization approver or involved in maintaining Bengali localization.  (there is no role name called localization maintainer as well - so far).


Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: e87da282-6ace-4f90-e894-b41c7e50649e
Version Independent ID: 7702ff20-f29d-321c-2516-92d448a6c333
Content: Distribute data globally with Azure Cosmos DB
Content Source: articles/cosmos-db/distribute-data-globally.md
Service: cosmos-db
GitHub Login: @SnehaGunda
Microsoft Alias: mjbrown

@lren Thank you for the detailed feedback. I have assigned this to the content author to evaluate and update as appropriate.

@lren  Thanks for pointing out the issue, I submitted a fix, the changes should be live soon. In future, if you see any minor updates or bug fixes, please feel free to submit a PR to the doc by clicking on the edit button at the right-hand corner. We encourage community contributions :-)

#please-close


Sync

:warning: Please install the  to ensure uploads and comments are reliably processed by Codecov.
Codecov Report
Attention: Patch coverage is 66.66667% with 1 line in your changes missing coverage. Please review.

Files with missing lines
Patch %
Lines

...rc/main/java/seedu/address/ui/StatusBarFooter.java
0.00%
1 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.

Files with missing lines
Coverage Δ
Complexity Δ

...seedu/address/logic/commands/AddClientCommand.java
62.50% <ø> (ø)
2.00 <0.00> (ø)

...edu/address/logic/commands/AddPropertyCommand.java
78.57% <ø> (ø)
7.00 <0.00> (ø)

...du/address/logic/commands/DeleteClientCommand.java
100.00% <100.00%> (ø)
2.00 <1.00> (ø)

.../address/logic/commands/DeletePropertyCommand.java
90.62% <ø> (ø)
14.00 <0.00> (ø)

...du/address/logic/commands/FilterClientCommand.java
89.47% <ø> (ø)
7.00 <0.00> (ø)

.../address/logic/commands/FilterPropertyCommand.java
89.65% <ø> (ø)
17.00 <0.00> (ø)

...java/seedu/address/logic/commands/ListCommand.java
100.00% <ø> (ø)
2.00 <0.00> (ø)

...rc/main/java/seedu/address/model/client/Email.java
92.85% <ø> (ø)
6.00 <0.00> (ø)

.../seedu/address/model/client/NameWithoutNumber.java
64.28% <ø> (ø)
6.00 <0.00> (ø)

...java/seedu/address/model/meeting/MeetingTitle.java
87.50% <100.00%> (ø)
8.00 <0.00> (ø)

... and 7 more


Dim Mode on Apple Cinema Display 27"

Required information

Mac model: 2016 MacBook Pro 15-inch with Touchbar
macOS version: 10.14.1 Beta (18B67A)
Monitor setup: Apple Cinema Display 27" with Apple Official USB-C to Thunderbolt Adapter

Description of issue / Feature request
External display does not dim despite being dimmable via keyboards and using all Apple products.
Debug Logs
12:42:06 AM : UI debugMode: true
12:42:08 AM : UI logToDisk: true
12:42:16 AM : <AerialView: 0x7f9c42151d60> startAnimation
12:42:19 AM : <AerialView: 0x7f9c42151d60> stopAnimation
12:42:22 AM : <AerialView: 0x7f9c42151d60> startAnimation
Logs to Disk uploaded via Cloud.app https://cl.ly/9d5b866336d9

Thanks for reaching out @stunzi
Can you clarify a few questions for me so I can have a look, since I don't have that setup:

Are you using both display (internal and external) simultaneously ?
Is your internal MBP monitor set as primary ?
When you use the dim keys, does it dim both screens simultaneously, or just the one that has the mouse/focus ?

If you indeed use both simultaneously

can you check if setting your external monitor as primary makes Aerial dim it ? (I'd expect in this case, the internal doesn't dim)

Thanks I'll wait for your feedback to have a look.

Tested Beta 6 and external Apple Cinema Display does not dim.
Tried Mirroring and both Laptop and External as primary.
Only thing I can think of is the Apple Thunderbolt to USB-C adapter.
Happy to run any sysdiagnostics or reports to help explore this further.
On Mon, Dec 03, 2018 at 8:54 AM, Guillaume Louel <<EMAIL_ADDRESS>> wrote:

Thanks for reaching out @ stunzi ( https://github.com/stunzi )
Can you clarify a few questions for me so I can have a look, since I don't
have that setup:

Are you using both display (internal and external) simultaneously ?

Is your internal MBP monitor set as primary ?

When you use the dim keys, does it dim both screens simultaneously, or
just the one that has the mouse/focus ?

If you indeed use both simultaneously

can you check if setting your external monitor as primary makes Aerial
dim it ? (I'd expect in this case, the internal doesn't dim)

Thanks I'll wait for your feedback to have a look.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (
https://github.com/JohnCoates/Aerial/issues/679#issuecomment-443717307 ) ,
or mute the thread (
https://github.com/notifications/unsubscribe-auth/AX21GG-Wr8dSbELI7WdMPK4-NggaE37Aks5u1S0MgaJpZM4Y96db
).

@stunzi there's nothing in beta6 yet about this, I was waiting on your answers. Can you answer my questions please :

When you use the dim keys, does it dim both screens simultaneously, or just the one that has the mouse/focus ?

It dims the primary display with the dock. In this case the external monitor. If I hold a modifier (func or opt I forget) it dims the secondary display.
On Wed, Dec 5 2018 at 8:05 AM, <<EMAIL_ADDRESS>> wrote:

@stunzi ( https://github.com/stunzi ) there's nothing in beta6 yet about
this, I was waiting on your answers. Can you answer my questions please :

When you use the dim keys, does it dim both screens simultaneously, or
just the one that has the mouse/focus ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub (
https://github.com/JohnCoates/Aerial/issues/679#issuecomment-444477985 ) ,
or mute the thread (
https://github.com/notifications/unsubscribe-auth/AX21GAzgqYCMOsnbl4cTU8WstPVMsU9iks5u18SBgaJpZM4Y96db
).


Support for using ToolbarAndroid in renderNavigationBar

We're unlikely to build this because we optimize for iteration speed and customizability, so we greatly prefer code in JavaScript. renderNavigationBar lets you return whatever you want as long as it conforms to the NavigationBar interface, so you could implement your own custom navbar that uses an Android component.

Looking over the code, routeMapper seems to have some hard assumptions, for example that the renderNavigationBar return value will render the title as a component rather than taking it as a text prop.


ApplicationDeployment is not included in .Net Core 3.1 or 5

The use of the ApplicationDeployment namespace (System.Deployment) is not included in .Net Core 3.1 or 5.0 .  I did not see a note stating this on the page, so I thought it might be good to include that for people wanting a way to programmatically update their WPF projects.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b4c30cb0-8e88-4277-bfb9-e32d85538ce2
Version Independent ID: 84b1bcd6-9554-b545-134f-ad66fb66e114
Content: Automatic app updates using ClickOnce deployment API - Visual Studio (Windows)
Content Source: docs/deployment/how-to-check-for-application-updates-programmatically-using-the-clickonce-deployment-api.md
Product: visual-studio-windows
Technology: vs-ide-deployment
GitHub Login: @Mikejo5000
Microsoft Alias: mikejo

Sorry @Mikejo5000 I don't know much about WinForms, WPF and ClickOnce these days. I mainly focus on web technologies.

@MrJohnLock Thanks for pointing this out. I've updated the articles referencing ApplicationDeployment class with a note. More updates coming with .NET 7.


Update pulumi-terraform to 65eb36cfeb

This PR updates pulumi-terraform to 65eb36cfeb, and re-runs code generation

Closed via #14


New Site:  watchseries.cr

Please add support for this site: watchseries.cr

youtube-dl does not support sites dedicated to copyright infringement.


Testing android sample

Codecov Report

Merging #22 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master      #22   +/-   ##
=========================================
  Coverage     94.19%   94.19%
  Complexity       14       14
=========================================
  Files             5        5
  Lines           310      310
  Branches         43       43
=========================================
  Hits            292      292
  Misses            4        4
  Partials         14       14

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 573b3ca...d1ab412. Read the comment docs.

Thanks man!


Add support for "session present" flag in io.netty.handler.codec.mqtt.MqttConnAckVariableHeader

Netty version: 4.1.0.Beta5
Context:
I'm implementing an MQTT broker based on netty using the classes from the package io.netty.handler.codec.mqtt, in the version 3.1.1 of the MQTT protocol it has been added a new flag in the variable header of the CONNACK message called "session present": http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718035, it would be great if that flag was supported in io.netty.handler.codec.mqtt.MqttConnAckVariableHeader.

We love contributions. Maybe care to submit a PR?

Am 29.07.2015 um 11:23 schrieb Jorge Castellanos Solaz<EMAIL_ADDRESS>Netty version: 4.1.0.Beta5
Context:
I'm implementing an MQTT broker based on netty using the classes from the package io.netty.handler.codec.mqtt, in the version 3.1.1 of the MQTT protocol it has been added a new flag in the variable header of the CONNACK message called "session present": http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718035, it would be great if that flag was supported in io.netty.handler.codec.mqtt.MqttConnAckVariableHeader.
—
Reply to this email directly or view it on GitHub.

@normanmaurer I tried to do it in PR. https://github.com/netty/netty/pull/4042
but I am not sure if it's OK to do it against 4.1 (the branch we're using) or if I should do it against master and you guys (or me?) take care of the cherry-pick
Also I just noticed now that I didn't format the commit properly, but it's so trivial I'm not sure it's worth it...

@flozano - Thanks for the PR! Doing against 4.1 should be fine. We will take care of cherry-picking into other branches.

@flozano thanks.. Adjusted commit message and cherry-picked into 4.1(a91e64850505938cfe3a2079e1ac2c247df6532b)  and master (6ff66fe967ed1aa45f2e87d263463f35c959b5cf).


fix(docs): update link to msgpsec

Fixe link to point to repo instead of current page anchor

woopsie, thanks!


Wraps updateHistory calls in documentReady

This allows 'setState' and 'replaceState' to be used in models without checking whether the document is ready.
Since documentReady runs callbacks on the next tick if the document is already ready, this does change behavior slightly. It seems like it's fine to wait a tick, but I haven't investigated thoroughly.
Related to #470, which was solved by documenting the fact that history changes don't work before the document is ready.

Whoops, it actually looks like https://github.com/yoshuawuyts/choo/pull/493 added this behavior. But this part of the documentation was missed?

Routing via pushState will not work until the DOMContentLoaded event has been fired.

Docs PR welcome if this was confusing (:
On Sun, May 28, 2017, 20:57 Tyler Murphy<EMAIL_ADDRESS>wrote:

Whoops, it actually looks like #493
https://github.com/yoshuawuyts/choo/pull/493 added this behavior. But
this part of the documentation was missed?
Routing via pushState will not work until the DOMContentLoaded event has
been fired.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/yoshuawuyts/choo/pull/504#issuecomment-304533242, or mute
the thread
https://github.com/notifications/unsubscribe-auth/ACWlev9eXFboD83NkKs-d6jyvpL6rwuTks5r-cODgaJpZM4NouDW
.


IndexError: list index out of range while running update-attack.py

While using workbench to integrate with attack website as per the guide, the following occurs while running update-attack.py
clean                  : ---------------------------------------- 0.00s
matrices               : ---------------------------------------- 7.74s
tactics                : ---------------------------------------- 0.13s
techniques             : ---------------------------------------- Running...Traceback (most recent call last):
File "update-attack.py", line 154, in
ptr'run_module'
File "/home/yash/Downloads/attack-website-master/modules/techniques/init.py", line 37, in run_module
return (techniques.generate_techniques(), techniques_config.module_name)
File "/home/yash/Downloads/attack-website-master/modules/techniques/techniques.py", line 53, in generate_techniques
check_if_generated = generate_domain_markdown(domain['name'], techniques_no_sub, tactics, side_nav_data, notes, deprecated)
File "/home/yash/Downloads/attack-website-master/modules/techniques/techniques.py", line 95, in generate_domain_markdown
generate_technique_md(technique, domain, side_nav_data, tactics[domain], notes)
File "/home/yash/Downloads/attack-website-master/modules/techniques/techniques.py", line 123, in generate_technique_md
technique_dict = generate_data_for_md(technique_dict, technique, tactic_list)
File "/home/yash/Downloads/attack-website-master/modules/techniques/techniques.py", line 239, in generate_data_for_md
tactic = [x for x in tactic_list if x['x_mitre_shortname'] == elem['phase_name']][0]
IndexError: list index out of range

Background: I used the predefined collection index and imported only the enterprise collection stix bundle while keeping everything the default.

Also, while importing the collection from the official ATT&CK repository on the frontend workbench, there was an object which could not be imported. Could this be a reason why the issue is occuring?

There is a strange issue, it all seems to work fine for some versions. But for those versions when I edit something on the workbench and then try update-attack.py, it stops working

For instance, It was working for the version 8 of enterprise matrix. But when I just tried to change the description of the enterprise matrix from the frontend and then tried to execute the update-attack.py, it shows the same error again.

Hi @Yash66666, thanks for opening this issue. The error looks related to the tactic of a technique. Did you modify a technique or was it only changing the description of the enterprise matrix?

Hi, It was just the description of the matrix. Also I was wondering is there a way to add or modify the assets in the ICS Matrix (this is related to workbench I think). The current assets are HMI, control server, etc. And finally wanted to ask about the ICS website to be integrated with the main Mitre website.

We are currently working on adding ICS to the website (See issue #272). We encountered errors on some ICS techniques similar to what you originally provided. If you are trying to integrate ICS technique to the website then it's likely to be that. We are working on making those fixing on the ICS STIX bundle itself. The target to add ICS to the website is during the next ATT&CK content release in April.
Regarding Also I was wondering is there a way to add or modify the assets in the ICS Matrix: Would appreciate if you could open an issue on https://github.com/center-for-threat-informed-defense/attack-workbench-frontend for the team to look into it

Sure thing, Thank you so much!