Unexpected linter output

Hey guys,
when running npx ts-standard --fix in my ts-project, I am getting:
Failed to load parser '@typescript-eslint/parser' declared in '--config » eslint-config-standard-with-typescript#overrides[0]': Cannot find module 'typescript'
Require stack:
-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/cli-engine/config-array-factory.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/cli-engine/cascading-config-array-factory.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/cli-engine/cli-engine.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/cli-engine/index.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/api.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/dist/ts-standard.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/dist/cli.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/bin/cmd.js: Error: Failed to load parser '@typescript-eslint/parser' declared in '--config » eslint-config-standard-with-typescript#overrides[0]': Cannot find module 'typescript'
Require stack:
-<EMAIL_ADDRESS>-<EMAIL_ADDRESS>- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/cli-engine/config-array-factory.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/cli-engine/cascading-config-array-factory.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/cli-engine/cli-engine.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/cli-engine/index.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/eslint/lib/api.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/dist/ts-standard.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/dist/cli.js
- /Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/bin/cmd.js
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:982:15)
    at Function.Module._load (internal/modules/cjs/loader.js:864:27)
    at Module.require (internal/modules/cjs/loader.js:1044:19)
    at require (internal/modules/cjs/helpers.js:77:18)
    at Object.<anonymous> (/Users/dominik/.npm/_npx/46101/lib/node_modules/ts-standard/node_modules/@typescript-eslint/typescript-estree/dist/parser.js:20:25)
    at Module._compile (internal/modules/cjs/loader.js:1158:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1178:10)
    at Module.load (internal/modules/cjs/loader.js:1002:32)
    at Function.Module._load (internal/modules/cjs/loader.js:901:14)
    at Module.require (internal/modules/cjs/loader.js:1044:19)

The error above looks to be failing because it cannot find your typescript module. Do you have typescript installed in the project you are running this on? ts-standard does not install typescript for you, but rather declares it as a peerDependency (meaning you have to install it manually yourself).

This is my package.json. So I guess, it should work.
https://github.com/Syy0n/serverless-blueprint/blob/master/package.json

Adding it as devDependency solved the problem. But via npx it is not working.

I kind of figured that would be the case (you already have it installed), but I like to start simple and work towards complex 😉
So I just pulled down that repo, and cd into the root directory (where package.json was). I then performed the following steps:

Deleted the //1, //2, etc... stuff in the package.json
Ran npm install
Ran npm run build
Ran npx ts-standard --fix

Everything worked great and ts-standard ran as expected. I also used a fresh node installation so that there were no globally install modules.
Try deleting your node_modules folder and redoing a full npm install all over again...sometimes things get screwed up in there.
It also might be the //1, //2 etc in your package.json as a lot of tools will read that file and if it does not contain valid JSON it might screw with how the tool operates.
If you are still running into issues, try recreating a minimal repo example with a single typescript file and a package.json with typescript and ts-standard and see if it continues to happen there.

Adding it as devDependency solved the problem. But via npx it is not working.

☝ That was super helpful. I forget that people run npx ... without actually installing the module they are running npx on. I just hit the same error. It is very interesting...will look into it, but I am guessing this is actually an issue with how @typescript-eslint/parser actually finds and uses the typescript package

After some additional research, it seems to be an issue with npx itself and how it resolves node_modules. From what I can tell, npx does not install the module globally or locally, but instead, in some weird cache location that does not have access (or maybe a bug) to either global or local node_modules.
For now, the only solution is to make sure ts-standard is installed locally before running npx ts-standard.
Interestingly enough, this issue might be solved by npm@7 because it will install peerDependencies by default thus, wherever npx installs ts-standard it should also install typescript too.


Sccp 304 testnet

btw, we can fix the simulate release build failure by, as it suggests, adding weth_address setting to the configs for optimism-mainnet


Add script that does total kops teardown

including all VPCs but not s3 buckets

Per our conversation: hopefully etc/testing/deploy/aws.sh --delete will be upgraded to delete cluster VPCs for you. Until that's done, this script might be helpful in making sure that the pachyderm clusters can be created


[nri-bundle] [newrelic-logging] feat: added support for templated cluster name value

Is this a new chart
No.
What this PR does / why we need it:
We're deploying the chart using ArgoCD and ApplicationSets with a cluster generator. To avoid having to bake NewRelic-specific logic into our wider ArgoCD Application generator, we need support for rendering cluster as a template variable in order to generate an appropriate value from generic values.
Which issue this PR fixes
This is a new feature.
Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[x] Chart Version bumped
[x] Variables are documented in the README.md
[x] Title of the PR starts with chart name (e.g. [mychartname])

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


remove time travel from paragraph

It gives 404, time travel should not be in paragraph, should be only in landing page

Well, this removes it but does not add it exclusively to the landing page. Maybe we should just use an absolute path for the link?

@dvdsgl landing page already have such - https://github.com/Microsoft/join-dev-design/blob/master/docs/index.html#L67

Great catch! There is definitely an error after clicking on a time-travel button during time traveling. However this PR doesn't fix the issue, because it deletes the line you are referring to.
I agree with @dvdsgl that the best solution would be changing the href to https://microsoft.github.io/join-dev-design/time-travel and re-building the history.

Actually line wasn't there, it was added by one of the contributors and not sure why it got merged - https://github.com/Microsoft/join-dev-design/commit/dd4b17d62ed6d54415465760b1da0a3fe32c0c20

I am still confused to what you are referring to - it is on landing page. Do you want to remove it from pages in time-travel mode?

Yes, actually time-travel wasn;t there in paragraph - it was added in this commit - https://github.com/Microsoft/join-dev-design/commit/dd4b17d62ed6d54415465760b1da0a3fe32c0c20 which gives 404 error

I would recommend disabling the link in time-travel mode. You would likely implement this in docs/time-travel/{index.html,time-travel.js}.
I would expect the following to be true:

Time Travel link remains as-is when viewed from top-level page
In Time Travel mode, Time Travel link causes a message to display, like "Time travel cannot be opened within time travel mode."
In Time Travel mode, the appearance of the Time Travel ink is not affected.

Please reopen if you develop this! Sounds useful. Thank you!


Support for PreSearch

Is your feature request related to a problem? Please describe.
新功能是否与解决某个问题相关, 请描述
PreSearch is a web3 search engine that focuses on user's privacy and chatGPTBox hasn't supported it yet.
Describe the solution you'd like
你期望的新功能实现方案
Make chatGPTBox performs on PreSearch just like it performs on Google.
Additional context
其他
Thanks for the developer's hard work.

v2.4.9


Create search form component

What does this mean
Implement the search form component based on the design below.
Information

create a search form that can search which language

Important

N/A

Figma design

For more detailed information, access the full design on Figma: View Figma Design

Search form component is made!
in this commit
the code:
search.Svelte

<script>
	    // Reactieve zoekterm
        let searchTerm = "";
        export let data;
        // Reactieve filtering
        $: filteredLanguages = searchTerm
            ? data.languages.filter(language =>
                language.name.toLowerCase().includes(searchTerm.toLowerCase())
            )
            : data.languages;
</script>

	    <!-- Zoekveld -->
        <input
        type="text"
        id="searchbar"
        placeholder="Search language"
        class="search"
        bind:value={searchTerm}
        />

<style>
.search{
    background-image: url(/icons/search-icon.svg);
    background-repeat: no-repeat;
    background-position: 2% 50%;
    padding: 13px 30px;
    border-radius: 10px;
    border: none;
    width: 85%;
}
</style>

Screenshots


Password Secret Type

A Password type needs to be created and implements the KeyMgr interface.
pkg/generator/password.go should add a struct and port the generate command and add the other methods required.

completed in #63


Errors using Unity 2021.3.25f1

I have two errors from the get-go:
error CS0117: 'RenderingUtils' does not contain a definition for 'ReAllocateIfNeeded'
error CS1061: 'ScriptableRenderer' does not contain a definition for 'cameraColorTargetHandle' and no accessible extension method 'cameraColorTargetHandle' accepting a first argument of type 'ScriptableRenderer' could be found (are you missing a using directive or an assembly reference?)
I am using the third person template and don't have enough low-level knowledge on the topic, could you please help me?

Those errors are caused because your Unity + URP version doesn't have those APIs.
You're using Unity 2021.3.x, which I'm no longer supporting.  The last version of Anaglyph3D to support Unity 2021.3.x was v2.1.1.  If you installed via the package manager, you can link to a specific package version by opening up your project's manifest.json and adding #v2.1.1 to the end of the git link.
"com.developedwithlove.anaglyph" : "https://github.com/ryanslikesocool/Anaglyph3D.git#v2.1.1"

It works! Thanks so much!


[BUG] test_in_set fails on A30 when DATAGEN_SEED=1698940723

I cannot repro this locally with my RTX6000, or on T4. This fails on A30 so far.
 I still can reproduce it on my local Quadro RTX 6000 (after removing the line @datagen_overrides).

This looks like a bug in our isin operator that incorrectly handles NaN. I can constantly reproduce it with any seed values but using data_gen= DoubleGen(special_cases=[float('nan')]).

Indeed, our code is correct. This is a bug in Spark: https://issues.apache.org/jira/browse/SPARK-36792.

When checking a isin (list...) with list size more than 10 elements (so the IsIn operator is called), Spark CPU cannot find NaN in the given list so it produces incorrect output. If the given list has 10 elements or less then the operator In is called and can correctly produce the answer.
scala> df.show(false)
+------------------------+
|a                       |
+------------------------+
|-1.8267391899860874E80  |
|-4.1675090531674384E-103|
|NaN                     |
+------------------------+

scala> spark.sql("select * from df1 where a in (CAST(-1.8267391899860874E80 AS DOUBLE), CAST(-4.1675090531674384E-103 AS DOUBLE), CAST('NaN' AS DOUBLE), CAST(-8.559820088589135E179 AS DOUBLE), CAST(1.1176123717679094E-166 AS DOUBLE), CAST(3.2567422266986805E-294 AS DOUBLE), CAST(-4.142554702629836E-92 AS DOUBLE), CAST(2.813265202703976E56 AS DOUBLE), CAST(3.4452000335727635E-205 AS DOUBLE), CAST('-Infinity' AS DOUBLE))").show(false)
+------------------------+
|a                       |
+------------------------+
|-1.8267391899860874E80  |
|-4.1675090531674384E-103|
|NaN                     |
+------------------------+

scala> spark.sql("select * from df1 where a in (CAST(-1.8267391899860874E80 AS DOUBLE), CAST(-4.1675090531674384E-103 AS DOUBLE), CAST('NaN' AS DOUBLE), CAST(-8.559820088589135E179 AS DOUBLE), CAST(1.1176123717679094E-166 AS DOUBLE), CAST(3.2567422266986805E-294 AS DOUBLE), CAST(-4.142554702629836E-92 AS DOUBLE), CAST(2.813265202703976E56 AS DOUBLE), CAST(3.4452000335727635E-205 AS DOUBLE), CAST('-Infinity' AS DOUBLE), CAST(-2.0147996288524902E268 AS DOUBLE))").show(false)
+------------------------+
|a                       |
+------------------------+
|-1.8267391899860874E80  |
|-4.1675090531674384E-103|
+------------------------+

Note that df1 above is read from parquet generated in the failed test. When I tried to create it manually then the InSet produces a correct output, so I guess the NaN value read from parquet here is probably a negative NaN.

should be closed by https://github.com/NVIDIA/spark-rapids/pull/9928


vim-mode: Unable to switch to normal mode in Firefox 57

On Firefox 57, once entering "insert" mode, it is impossible to enter normal mode again (neither Esc nor Ctrl-[ works). It works well on Firefox 56.

I can't reproduce this in Firefox 57 Linux. Could you try to debug what's going on? I.e. is there an error in the console? What is cmKey in keymap/vim.js being called with when you press escape? Etc.

Hah, I found the reason is that I installed Vimium for Firefox... Thanks!

This is broken again. Neither esc nor control+[ work.

For me the issue was caused by Vim Vixen. Disabling it on the page was in vain, only works when I disable it completely :/


docs/mixin: add variables to promscale dashboard

Description

This PR is improving dashboard in the following ways:

Overview and Database rows now contain panels. Previously they were empty in code which caused UX issues with row collapsing
Maintenance jobs graphs are moved to an additional section
Datasource can be now selected instead of using only one. As a side effect, we have only one place in code to set datasource UID.
All queries have namespace selector which defaults to .+. This improves workflow in kubernetes environments with multiple promscale-connectors. It doesn't affect environments without namespace label key present.

Merge requirements
Please take into account the following non-code changes that you may need to make with your PR:

[ ] CHANGELOG entry for user-facing changes
[ ] Updated the relevant documentation

Since this is a JSON dump, it will be almost impossible to review (we should really consider moving this to jsonnet), so I am attaching screenshot of the dashboard


DrvFs: compiler reports file missing, but it is present

On build 17046, while compiling the GCC cross-compiler, it bailed out with a "No such file or directory" error when referencing builtins.def — but the file is present. Could this be a variant of #2448 or #2464?
The error:
g++ -fno-PIE -c   -g -O2 -DIN_GCC  -DCROSS_DIRECTORY_STRUCTURE   -fno-exceptions -fno-rtti -fasynchronous-unwind-tables -W -Wall -Wno-narrowing -Wwrite-strings -Wcast-qual -Wmissing-format-attribute -Woverloaded-virtual -pedantic -Wno-long-long -Wno-variadic-macros -Wno-overlength-strings   -DHAVE_CONFIG_H -I. -I. -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/. -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/../include -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/../libcpp/include -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/bld/gcc/./gmp -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gmp -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/bld/gcc/./mpfr/src -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/mpfr/src -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/mpc/src  -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/../libdecnumber -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/../libdecnumber/bid -I../libdecnumber -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/../libbacktrace -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/bld/gcc/./isl/include -I/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/isl/include  -o tree-streamer-in.o -MT tree-streamer-in.o -MMD -MP -MF ./.deps/tree-streamer-in.TPo /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/tree-streamer-in.c
In file included from /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/tree.h:23:0,
                 from /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/tree-streamer.c:27:
/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/tree-core.h:213:24: fatal error: /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/builtins.def: No such file or directory
compilation terminated.
Makefile:1099: recipe for target 'tree-streamer.o' failed
make[1]: *** [tree-streamer.o] Error 1
make[1]: *** Waiting for unfinished jobs....

But the file reported as missing is present.
ls -l /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/builtins.def
-rwxrwxrwx 1 root root 83663 Dec  1 00:38 /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/builtins.def

After a clean re-start, the compilation has failed again, exactly as above, but with a different file:
/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/gcc/builtins.def:1001:29: fatal error: chkp-builtins.def: No such file or directory
But the file is alive and well, where it is supposed to be.

After a further clean re-start, the same compilation fails in a different place. A file appears to be missing, but it is safely in the right place when found with ls:
In file included from /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/libgfortran/libgfortran.h:44:0,
                 from /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/src/gcc-7.2.0/libgfortran/generated/pack_c16.c:26:
/mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/bld/gcc/gcc/include/stddef.h:1:15: fatal error: stddef.h: No such file or directory
 #include_next <stddef.h>
               ^~~~~~~~~~

but:
$ ls -l /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/bld/gcc/gcc/include/stddef.h
-rwxrwxrwx 1 root root 14165 Dec  1 23:08 /mnt/e/Users/john/Documents/ffmpeg/MultimediaTools-mingw-w64/sandbox/bld/gcc/gcc/include/stddef.h

@Warblefly
The workaround posted in this link fixes this problem temporarily.
https://github.com/Microsoft/WSL/issues/2448#issuecomment-342254181
It worked for me. I'm on build 16299.64. Need to see if the cumulative patch of 16299.98 fixes this.

Thank you for replying. I'm afraid this workaround, which I've already tried, is unsuccessful in build 17046.
It is a slightly different problem: the file is occasionally simply "not found"; this isn't an EINVAL return.

#2484 (sic).

Well spotted, @therealkenc.

@Warblefly Seems like you got all the compilation issues one can get. Lel
Sorry to hear it doesn't fix your problem. My theory is its a fallback as the command states. It remounts the drive with a previous version of the DrvFS driver. Since I'm on 1709 16299.98, the fallback driver for me is the 1607 driver module. Which works great.
Maybe for you since you are beyond 1709, your fallback is the faulty one present in 16299 because its the current stable one.
Anyway my issue was this one: https://github.com/Microsoft/WSL/issues/2464#issuecomment-347217184
I just posted in related threads to see if someone knew about it because there was no specific issue for it until now.
@therealkenc The cumulative update to 16299.98 fixes this issue. But it seems performance has taken a hit. Kernel compile went from 240~ with ccache to ~350 with ccache.

Thanks for the info @nutcasev15 — indeed, whether or not the DrvFs filesystem is mounted with, or without, the "fallback" workaround given some time ago, the fault occurs. It makes the job impossible, unfortunately.

@Warblefly are you on the latest, per #2448  : " The November 30th, 2017 Cumulative Update (KB4051963) for the Fall Creator's Update contains the fix for this issue. Please install this update, and let us know if you encounter any further issues. To verify if you have installed the update, your OS build number should be 16299.98 or later."

@AnneTheAgile Thank you for asking these questions. I am happy to offer this information:
(1) The build is 17064.rs_prerelease.171118-1403.
(2) It is difficult to say precisely at this point, but there was an earlier build on which my compilation did work correctly (other than the occasional instance of "fork" not working as reported in #2469)

A few members of my team (myself, @rajsesh, @yiyang-msft) are seeing this as well, on builds in the 1705x-1706x range.

Correction: I mis-typed the build. Here, it's 17046.

this can be "reproduced" fairly "consistently" in my setup in several completely different use cases, see (duplicate) issue @Warblefly linked above.

Isssue still exists in 17093. I tried to create simple program, which simulates this behavior. On DrvFS reproduces problem, on VolFS works without errors.
`
#include <stdio.h>
#include <pthread.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
void *thread(void *arg)
{
long num = (long)arg;
while(1)
{
	int handle;
	const char *fileName = (num % 2) == 0
			? "/mnt/d/Linux/1/Test.c"		// existing file
			: "/mnt/d/Linux/2/Test1.c";		// nonexisting file in nonexisting directory

	handle = open(fileName, O_RDONLY);
	if (handle == -1)
	{
		if ((num % 2) == 0)		// existing file
		{
			printf("Existing file open error %d !!!!\n", errno);
		}

		continue;
	}

	close(handle);
}

}
#define THREAD_NUMBER	16
int main(void)
{
pthread_t t[THREAD_NUMBER];
long i;
for (i = 0; i < THREAD_NUMBER; ++i)
{
	if (pthread_create(&t[i], NULL, thread, (void *)i) != 0)
	{
		printf("Thread creation error\n");
		return 0;
	}
}

while(1)
{
	sleep(0);
}

}
`

in my scenarios, the error also happens on VolFS, albeit not as often as on DrvFS

I think It is a problem with make -j X ?

We believe we have found the cause of this issue, and are working on a fix. We're trying to get this fix into the upcoming Spring Update as well (probably as an update after release).

that is great news, thanks sven!

@SvenGroot is this issue a confirmed duplicate of https://github.com/Microsoft/WSL/issues/2780? I just saw your comment. It would be so nice to see a fix for https://github.com/Microsoft/WSL/issues/2780 soon.

We're trying to get this fix into the upcoming Spring Update as well (probably as an update after release).

Can you confirm this? :) I'd like to opt-out from the Insiders channel now. (AFAIK you need to do the Insiders version matches the stable version?)

@donaldpipowitch as the fix hasn't yet made it into the skip-ahead builds, you can opt out now to be not worse off. I'm sure a possible backport into the stable builds will be announced or negated here once a fix has made it into a future insider build.

Wow. Cool, Thank you. Dumb question: will this come to stable in the "April" update or in the next big update in about six months?

@donaldpipowitch - We will need to bake the fix in the insiders for sometime and get some confirmation that it addresses the issue. Then we can try to backport this to SCU ("April") release.
@heldchen - Just to be clear, this is still happening for you on 17655?

@sunilmut yes, debian on 17655

Thanks @heldchen. Adding @SvenGroot

@heldchen Do you have a repro we can try? It sounds like you're hitting a related, yet somehow different, issue.

@SvenGroot it's basically the magento part from #2780 that still fails, the other cases have been resolved for me in 17655. it did fail very often fail on read errors before 17655, whereas in 17655 the failure seems to be reduced to the file delete action.
creating a simple enough repo isn't easy unfortunately due to it happening in a largeish client's project, and only after roughly 1min of the 3min compilation process which creates around 20k new files: It is happening during the frontend theme compiling where 3 (php) process run in parallel, with one of them hitting the issue when trying to cleanup a lock file created in the same process a few seconds before.
adding strace into the mix unfortunately makes the error go away. probably because the build process takes 3x longer and with that somehow avoids the threading issue?

Did the issue reported here (specifically David's test case) exist in Fall Creators 16299, or was it introduced after?

my issues started shortly after 16299, first noticed it in build 17040).

Can those who still see this issue try again using build 17677, and report the results here?
Thanks!

I've managed to get a strace by replacing $result = unlink($file) with exec("ls -al $file && strace -o /tmp/unlink.trace -f rm $file", $output, $result);:
$output
-rw-rw-r-- 1 MyCustomer MyCustomer 10 May 25 01:50 /mnt/t/MyCompany/Projects/clients/MyCustomer/src/www/var/tmp/alternative-source-css.lock

(just as proof that the file actually exists before trying to delete it)
unlink.strace
4027  execve("/bin/rm", ["rm", "/mnt/t/MyCompany/Projects/clients"...], [/* 29 vars */]) = 0
4027  brk(NULL)                         = 0x7fffd4e84000
4027  access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
4027  mmap(NULL, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3afb640000
4027  access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
4027  open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 6
4027  fstat(6, {st_mode=S_IFREG|0644, st_size=51925, ...}) = 0
4027  mmap(NULL, 51925, PROT_READ, MAP_PRIVATE, 6, 0) = 0x7f3afb633000
4027  close(6)                          = 0
4027  access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
4027  open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 6
4027  read(6, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\0\4\2\0\0\0\0\0"..., 832) = 832
4027  fstat(6, {st_mode=S_IFREG|0755, st_size=1689360, ...}) = 0
4027  mmap(NULL, 3795296, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 6, 0) = 0x7f3afb060000
4027  mprotect(0x7f3afb1f5000, 2097152, PROT_NONE) = 0
4027  mmap(0x7f3afb3f5000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 6, 0x195000) = 0x7f3afb3f5000
4027  mmap(0x7f3afb3fb000, 14688, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f3afb3fb000
4027  close(6)                          = 0
4027  mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f3afb630000
4027  arch_prctl(ARCH_SET_FS, 0x7f3afb630700) = 0
4027  mprotect(0x7f3afb3f5000, 16384, PROT_READ) = 0
4027  mprotect(0x7f3afba0e000, 4096, PROT_READ) = 0
4027  mprotect(0x7f3afb623000, 4096, PROT_READ) = 0
4027  munmap(0x7f3afb633000, 51925)     = 0
4027  brk(NULL)                         = 0x7fffd4e84000
4027  brk(0x7fffd4ea5000)               = 0x7fffd4ea5000
4027  open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 6
4027  fstat(6, {st_mode=S_IFREG|0644, st_size=2011712, ...}) = 0
4027  mmap(NULL, 2011712, PROT_READ, MAP_PRIVATE, 6, 0) = 0x7f3afb437000
4027  close(6)                          = 0
4027  ioctl(0, TCGETS, {B38400 opost isig icanon echo ...}) = 0
4027  newfstatat(AT_FDCWD, "/mnt/t/MyCompany/Projects/clients/MyCustomer/src/www/var/tmp/alternative-source-css.lock", {st_mode=S_IFREG|0664, st_size=10, ...}, AT_SYMLINK_NOFOLLOW) = 0
4027  geteuid()                         = 2103
4027  newfstatat(AT_FDCWD, "/mnt/t/MyCompany/Projects/clients/MyCustomer/src/www/var/tmp/alternative-source-css.lock", {st_mode=S_IFREG|0664, st_size=10, ...}, AT_SYMLINK_NOFOLLOW) = 0
4027  faccessat(AT_FDCWD, "/mnt/t/MyCompany/Projects/clients/MyCustomer/src/www/var/tmp/alternative-source-css.lock", W_OK) = 0
4027  unlinkat(AT_FDCWD, "/mnt/t/MyCompany/Projects/clients/MyCustomer/src/www/var/tmp/alternative-source-css.lock", 0) = 0
4027  lseek(0, 0, SEEK_CUR)             = -1 ESPIPE (Illegal seek)
4027  close(0)                          = 0
4027  close(1)                          = 0
4027  close(2)                          = 0
4027  exit_group(0)                     = ?
4027  +++ exited with 0 +++

For the lulz I ran David's test case on 17672 and can confirm that much seems okay now.

@therealkenc as said in a few comments before and shown in the above strace, the issue still exists for delete actions when there is lot of I/O and concurrency. the test case does only use open actions which I can confirm are fixed for me as well.

Repetition is not constructive.

just wanted to be helpful by pointing out that the test case only tests one part of the problem, or one problem in a family of problems and that the closure of this issue might have been premature.
but before I start "repeating" myself again: where do you want me to continue with my unlink-while-heavy-concurrency-i/o: new issue? #2780?

@heldchen - the ESPIPE error you are seeing looks like it is on stdin (fd=0) so is likely unrelated. If you can get a repro where one of the other file system commands fails it would be great if you could share (we can route depending on what the error is, and if there is already an existing issue covering it). Unlink on /mnt/x (DrvFS) has had a few reported issues but has been improving over time.

thanks @Brian-Perkins for confirming the ESPIPE is not relevant.
turns out @therealkenc was right to close this issue after all. I spent some more hours logging all threaded unlink-actions using strace and comparing them. it looks like the Magento framework using a file-based self-rolled locking mechanism. when one thread acquires a lock. unfortunately they did not account for multiple threads trying to acquire the lock at the same time, so in some rare circumstances more than one thread would think that it owns the lock. in these cases, according to my strace logs the chances are actually quite high that two threads try to release their lock by unlinking the file at virtually the same time. in these cases, PHP's file stats cache seems to momentarily reports outdated info to the losing thread, thus resulting in the "errors" I've been observing.

I guess this manifests is 17134 (1801 RTM).

@sunilmut thanks for the fix!
Will this bugfix get backported to 1803?

@jowadmax - Once we get some kind of confirmation (from the community) that the issue is fully resolved by the fixes, yes, the fix will be considered for backporting to 1803.

@sunilmut - Hello and thanks for the fix !
I am also impacted by this issue after1803 update and this is very restrictive on my job. I am not sur I can wait the patch. Do you know if a fresh installation (reinstall Windows) can solve this problem ?

@sunilmut Is there a way to install the insider build without permanently enabling preview builds? I might be able to convince my company to let me install one preview build so I can check if the fix works, but I sure don't want to get preview builds all the time.
(I'm seeing this problem many times during one build job, i.e. I have to re-start the build a dozen times or so until it finally succeeds. So if it's fixed, I should be able to tell with good accuracy.)

Can confirm this issue is fixed for me on build 17692. Would love to have this backported to 1803 so I don't have to leave my work machine on a Insider build.

@jowadmax Thanks for the tip, unfortunately I don't have enough space for another Windows installation in a VM on my system.

Not working, my wsl.conf is correct:
$ cat /etc/wsl.conf
[automount]
options = case=off

Yet I get random error so WSL is unusable for me...

@sunilmut     A backport would be great. My 1803 is frustrating me.

@therealkenc Seeing this in 17134 (1803), confirming your suspicion. A backport or workaround would be wonderful as I'm unable/unwilling (requirement of Microsoft Account/non-system local account) to install one of the insider builds.

Yeah the December OP reported 17046 (before 17134) so there was no real reason to expect it to work on the April Update. My marking it insidertransient for a few days was just "wrong" on my part; apologies. It was reported that 16299 is okay, fueling the incorrect assumption. At the time it was somewhat confusing as to what broke when, what was being fixed precisely, and in what release (#2484 still flaps in the wind uncategorized).
Beyond that, four asks for a backport is now three too many. I don't have any specific insight on the interworkings of the backport process (and, I get the impression it is somewhat opaque to the WSL devs as well and not entirely in their control). But, given that it was fixed in late May, this is mid-August, and the Redstone 5 (1809) release is already starting to be locked down for October... if it were me, I wouldn't place a large bet on a QFE. Who knows. Could happen. Either way, rest assured all the devs get all the issue updates whether you ping them or not.


Relative paths to assets (images, scripts, links) in Readme are broken

https://dimitrisnl.github.io/wifi-connect/

The how it works image has a relative path.
We should identify paths without domain.

https://github.com/dimitrisnl/wifi-connect

On second thought, this should be handled on Scrutinizer instead. Keeping the issue open when we upgrade to a version that has proper absolute links in Readme.
https://github.com/balena-io-modules/scrutinizer/issues/58

Tracking progress here https://github.com/balena-io-modules/scrutinizer/pull/59


Request #8294435

Original: "Show Toolbar"
Your request: "Voir la barre d'outils"

This should have been closed in https://github.com/Imaginaerum/magento2-language-fr-fr/commit/a1591eb23c46bc63374dbab7c4c342998b6bb273.


duplicate declaration

Deployed to: http://staging.quran.com:32768

@infacq Thank you for the pull request.


Pattern keyword and unicode option in repl/cli

Fix #143
bors r+
Pattern keyword is just a placeholder, will be checked once #170

bors r+


new addconsumers methods

Pull Request Reason:
Need to filter when collectively injecting consumer classes
Scenario:
In my project there are a lot of Consumer and ConsumerDefinition classes. These consumers listen and consume the messages coming from two bus instances. Therefore, I injected multiple buses. To configure the buses I used busConfigurator.AddConsumers(myAssemblyCollection); method. However, at this point I have a special case which requires a consumer which should be bound to only one bus. MassTransit currently cannot filter the consumers during injection. As a result; when I use multiple busses and want to inject the consumers collectively, I am not able to do that.
My offer is,
Below extention methods already exist:
AddConsumers(this IRegistrationConfigurator configurator, params Assembly[] assemblies)
AddConsumers(this IRegistrationConfigurator configurator, params Type[] types)
Overload methods could be created with filter parameter such as below:
AddConsumers(this IRegistrationConfigurator configurator, Func<Type, bool> filter, params Assembly[] assemblies)
AddConsumers(this IRegistrationConfigurator configurator, Func<Type, bool> filter, params Type[] types)
By using these methods, I can configure my second bus like below:
busConfigurator.AddConsumers(myAssemblyCollection, type => ! type.Namespace.StartWith("XXX.OnlyForFirstBus"));

Cleaned, rebased, merged.

Cleaned, rebased, merged.


Invalid URL

It worked fine till I try to change config for using groq through OpenAI_Endpoint

what did you use as the config value?

what did you use as the config value?

groq token and endpoint as 'groq'

the endpoint needs to be a URL. For Groq it would be https://api.groq.com/openai/v1

the endpoint needs to be a URL. For Groq it would be https://api.groq.com/openai/v1

Thanks, I will check


How about add new function to clean mapping data

Due to an excel file contains many sheets, there exists same property but shows as different header name.
Example:
When Sheet 1 .

Column 1
Column 2
AMOUNT

Text
Text
9

excel.addMapping<ClassA>(A=>A.RATE,"AMOUNT");
...
Sheet 1 done.
Next Sheet 2.

Column 1
RATE
Column 3

Text
8
Text

Due to header have RATE which equal to ClassB.
I think it will auto mapping.
After use excel.worksheet<ClassB>(Sheet2)
ClassB's RATE is null due to it's mapping to "AMOUNT".
This is what i encounter.
Is it possible add new function to clean the certain one or all mapping?

The "easy" way to do this now would be to just re-initialize excel again with excel = new ExcelQueryFactory("excelFileName"); which would have a result similar to clearing all existing mappings.

Thank for this tool. It save me a lot of time about processing Excel file.
And I encounter the problem like this one.
In current practice, it take almost 4 secs to execute new ExcelQueryFactory  each time.
My Sample like below.
For Sheet 1
excel.addMapping<ClassA>(x=>x.Property,"Column A",x=>{ logic  A });
For Sheet 2
excel.addMapping<ClassB>(x=>x.Property,"Column A",x=>{ logic   B }); // this will cause error Exception {"An item with the same key has already been added."}
Sheet 1 and Sheet 2 have same column but with different logic.
And this kind of situation happens very often in my case.
Although it's ok to use new ExcelQueryFactory.
but I think it's not a long term solution due to the high cost.

Another option might be to do the mapping using annotations which were added in #34.
The issue experienced here sounds similar to another issue fixed by #90. I will look into it.


Specify width

Due to a bug in Select2 you need to specify width when a select2 element is hidden on render else it will be 100px wide.
select2/select2/issues/3278
But afaik there is no option to specify width...

Fixed by your PR.


Allowing Subclassing of Pickler

Adding to allow for custom pickle implementation (working on implemeting a lighter-weight streaming version of pickle)
Simply making private variables protected to allow for subclasses.

Can you add some clues as to why you need direct access to these private fields?  Can't you do it via virtual methods?

In general, I found I could not effectively subclass Pickle without access to these fields.
For example, it is not apparent to me how I can effectively set the private OutputStream out such that when calling the public void save(Object o) method will not result in an Exception.
Is your suggestion to expose these fields as getters/setters rather than making the fields protected or did you have something else in mind?

I was thinking you could perhaps simply override the save method.

Thank you for the comment.
I think there is a basic issue with trying to override the save method:
It appears to me not possible to subsequently use a bulk of the already defined methods / implementation.
For example, if I override save:

I cannot use or rely on the Pickler to call the dispatch method.
Memoizing cannot work without a custom implementation

In essence, I would essentially be copying the Pickler, but really only need to override the dump method or implement a similar method to dump and have the variables be protected.

And registering a custom pickling class is not enough to achieve your goal either?
I don't really mind to make things public (though for abstraction's sake I'd rather keep stuff private as much as possible) but it's a bit hard to see what you need to do in your situation without seeing the actual code you're writing.
Would it be possible and useful to share your version of the pickler once it's finished?

It seems that registering a custom pickling class is not enough because we want to be able to separate the objects in a streaming scenario.
In this case, note I am only suggesting that some of this fields become protected to allow for subclassing, not to lift these to public.
I can send you an anonymized code snippet if that would help in understanding the use case.
Thanks.
Ben

While not being used, I've made similar changes to the .net version of the pickler. I'd like them to remain in sync.

Thank you.
Would it be possible to create a release with these changes?

I suppose so, are the current changes enough for your cause?

Yes . The current changes seem to be enough for what we are working on.
Thank you.

Release 4.13 has been uploaded to Nuget and Maven.


Frequent Issue with Delete Key Not Functioning After Upgrading to Latest Version of VS Code

Type: Bug
Description
I recently upgraded VS Code to the latest version and encountered a recurring issue: the delete key doesn't function to delete characters. I have confirmed that the delete key works properly in other applications, indicating that this issue is specific to VS Code.
Steps to Reproduce
Open VS Code.
Input some text in the editor.
Attempt to delete characters by pressing the delete key.
Expected Behavior
Pressing the delete key should delete the character at the current cursor position.
Actual Behavior
In some cases, pressing the delete key does not delete characters.
 I have confirmed that the delete key works properly in other applications, indicating that this issue is specific to VS Code.
Steps to Reproduce Open VS Code. Input some text in the editor. Attempt to delete characters by pressing the delete key. Expected Behavior Pressing the delete key should delete the character at the current cursor position.
Actual Behavior In some cases, pressing the delete key does not delete characters.
VS Code version: Code 1.89.1 (dc96b83, 2024-05-07T05:13:33.891Z) OS version: Windows_NT x64 10.0.19045 Modes:
System Info
Extensions (47)
A/B Experiments

I rolled back to version 1.88.1 and everything is back to normal.


Add support for NULL keys

When reading an existing INI file, a NullReferenceException is thrown for empty keys. For instance:
[SECTION]
Name = ""

This file is not going to be loaded. Unfortunately, this is a real case scenario, and it would be nice to have the possibility to open the configuration anyway.

I was unable to reproduce this exception. The configuration is parsed fine.


What is the license?

Please add a license to the repo, thanks

Thanks @vaimdev , added file, it is MIT

@ivictbor , thanks.


Improve Useability of ACL Rules for Large Scale Projects (or long term maintanence)

ISSUE TYPE

Improvement Request

COMPONENT NAME

Improvement Request (UI, Functionality)

CLOUDSTACK VERSION

4.19

CONFIGURATION

N/A
OS / ENVIRONMENT

N/A
SUMMARY

As an admin, I find creating and manaing ACL Rules to be painful at the moment.
Here are a few ideas id like to propose to improve the useability for long term, and large scale projects.
1. Display ACL Rules in the form of a excel like table

Currently the UI for ACL Rules is in a 2x4 Column
It makes it hard to scan and read.
Also, depending on the input, the columns can be hidden/shown. Which makes it even harder to read as every rule can have a different arrangement.

2. Allow users to import ACL from a CSV File

As an admin, id like the ability to plan my ACL lists offline first in an excel.
Once ive finalised my list, i would like to be able to import it to cloudstack via CSV
This would greatly help my workflow (i can also keep the excel as a backup)

3. Allow users to Search ACL Rule

In large scale projects like ours, we have around 20-40 ACL Rules.
As the rules increase, its hard to read and identify.
Supporting search ACL Rule would help.

4. Display Tags in the UI for each ACL Rule

Tags are supported in the GUI, but they are not displayed.
If tags are shown in the UI, it makes it easier to scan

5. Support ability to Filter by Tags

This is self explanatory, makes managing ACL rules easier for large lists.

6. Allow user to create an ACL Rule for Multiple Ports

I remember reading somewhere that the logic is supported (Eg. Terraform Provider Docs)
But the UI doesnt seem to support it.
This will help greatly reduce the number of rules we create.

7. Export ACL List is missing some data

For example, 'Ports' are not available in the exported CSV when exported.
There may be others that are missing from the CSV output too.

8. Support ACL on Public IP

Good to have, adds more security

9. Support Destination CIDR

Good to have, adds more security
Reduces number of ACL Rules required

10. Option to set ACL Rule to be the very bottom by default

At the very end of our ACL Rule, we usually have a 'Deny All'
But when i use API or Terraform to create ACL Rules, the arrangement and sequence (including the rule number) gets all messed up.
Would be great if we could have a parameter and function in the UI to make the rule fix itself at the bottom of the list.

I know its a long list. Thank you for reading!
STEPS TO REPRODUCE

Experiencing this issue cause our projects are getting bigger and bigger.

EXPECTED RESULTS

Better experience using ACL Rules

ACTUAL RESULTS

Challenging to use ACL rules for large scale projects.

Created! Will close this ticket.

thanks @btzq , much appreciated.


(question) Confused by slowdown after using ShardWriter instead of TarWriter

Hello,
thanks for creating this tool. My setting is I have a dataset that does not fit in RAM and work on an NFS cluster with slow IO speeds (no SSDs).
I'm confused because when I train with the training split into shards (each is 3G) instead of a single tar I get an order of magnitude slowdown. I'd like to understand what webdataset does that causes this.
I thought that one point (among others) of webdataset was that because one uses an iterable dataset, one can load part of dataset, use that for generating batches of samples, and in the meantime load the next part in the background and switch to it once all samples in the first part have been used. To get around the issue of not being able to shuffle one can split the whole dataset into some files (shards), and shuffle those (as well us shuffling inside of a shard).
I don't understand why having to load the shards (files) would cause such a large slow down since there are not that many (~70). If it matters the data is saved in binary format (there is no encoder and the decoder just recovers the objects).
edit: Have I understood the code correctly that a tarfile is not actually read in one go but instead the contained files are read one by one?

I'm confused because when I train with the training split into shards (each is 3G) instead of a single tar I get an order of magnitude slowdown. I'd like to understand what webdataset does that causes this.

You are probably reading multiple shards from the same rotational disk drive (if you're using num_workers > 1, causing it to perform random seeks. The reason you are not seeing that when you use a single .tar file is that a single .tar file can't be split among multiple workers so only one worker tries to read it.
This is not a WebDataset limitation, it's a limitation of your hardware. That is, a rotational drive can deliver about 200 Mbytes/s when you read only a single shard from it, but if you use any I/O pattern that results in random access (e.g., multiple shards, individual files, LMDB, etc.), that can drop to 20 Mbytes/s. To get the maximum I/O bandwidth of 200 Mbytes/s from a single rotational drive, it's best to use num_workers=1.
If you have multiple physical drives available, you can distribute shards across them and WebDataset will read in parallel from multiple drives. If  you're using NFS as your network file system, obviously that requires some bookkeeping on your part.
The AIStore object store is an alternative that makes this easier.

Have I understood the code correctly that a tarfile is not actually read in one go but instead the contained files are read one by one?

That's correct, and it's usually the right behavior for WebDataset. However, in your case, you may benefit from reading the data as quickly as possible; you can achieve this by using something like:
url = "pipe:mbuffer -m 4G /nfs/dataset/train-{000000..000999}.tar"
ds = wds.WebDataset(url, ...)

You might even do something like this:
url = "pipe:flock /tmp/shard.lock -c 'cat /nfs/dataset/train-{000000..000999}.tar' | mbuffer -m 4G"
ds = wds.WebDataset(url, ...)

This will use a lock to ensure that only one worker at a time accesses the NFS directory. It will then read the shard as quickly as possible and buffer the data in memory.

Please let me know if that fixes your issue.

Thank you very much for the elaborate answer. I will try your suggestion out!

Hi @tmbdev looking at webdataset again and have a question about IO in the context of webdataset. Can one get a performance speedup just because of doing IO sequentially instead of randomly, but while still doing the same number of read calls? I had always understood that the benefit from sequential IO was that, since there's no ambiguity as to what data will be read next, you can always do read operations that read in a lot of data at once (effectively reading multiple files at once), instead of reading file by file. But based on the default implementation of webdataset (which reads file by file) and what you talk about here, apparently there is a benefit to doing sequential IO even if one is not "batching" the read operations?

There are two kinds of files we're talking about: the shards and the sample files contained within the shards. WebDataset reads many sample files with a single read() call, and it reads all those files sequentially.
The benefits from sequential reads occur at several levels. At the level of disk hardware, sequential reads avoid the head seeking behavior (or analogous overhead on SSD's). At the level of network file systems, sequential reads can be streamed from the file server without any round trips.

WebDataset reads many sample files with a single read() call

Doesn't that contradict your earlier statement?

Have I understood the code correctly that a tarfile is not actually read in one go but instead the contained files are read one by one?

That's correct, and it's usually the right behavior for WebDataset.


[sival,pwrmgr] Fix chip_sw_pwrmgr_sysrst_ctrl_reset test

This has two simple commits:

Fix test C-code to read flash after configuring it.
Fix the corresponding testpoint description.

The failing test (//sw/host/ot_certs:ot_certs_test) is unrelated and doesn't appear to have good controls for reproducing:
thread 'x509::tests::offsets_are_correct' panicked at 'could not create generic certificate: trying to register 31 bytes for variable Variable { name: "cert_signature_r", convert: None } but the the user-provided settings would generate 32 bytes', sw/host/ot_certs/src/x509/tests.rs:63:31

Merging looks okay to me!


[WIP] Routing

This is experimental but I make that available for extension / discussion purpose.
Thanks to @AdSchellevis for the proxy code which is a good template for own projects.
Note: The template still may require changes to work on other devices than an APU1d4. It may be a good idea to check it with the interface names removed.

@fichtner I think it can be merged for now (for first tests)

If you want to rename the directory, you can do that if you like

Merged, thanks!

@fichtner Let's see how long it takes until somebody will blame me for bad web design ;)

you'll do just fine, no worries at all. I am happy for this plugin. :)


Update to taxcalc 0.13.0+

This PR resolves #733 and #735.
TODO:

[x] Miscellaneous variable renaming
[x] Front-end API changes
[x] Back-end API changes (flask_server.py/celery_tasks.py focused changes)

@martinholmer would you mind giving me a mapping from the old results keys to the new results keys?
Old:
{'cdf_bin',
 'cdf_dec',
 'df_bin',
 'df_dec',
 'fiscal_tot_base',
 'fiscal_tot_diffs',
 'fiscal_tot_ref',
 'mX_bin',
 'mX_dec',
 'mY_bin',
 'mY_dec',
 'pdf_bin',
 'pdf_dec'}

New:
{'aggr_1',
 'aggr_2',
 'aggr_d',
 'diff_comb_abin',
 'diff_comb_adec',
 'diff_comb_xbin',
 'diff_comb_xdec',
 'diff_itax_abin',
 'diff_itax_adec',
 'diff_itax_xbin',
 'diff_itax_xdec',
 'diff_ptax_abin',
 'diff_ptax_adec',
 'diff_ptax_xbin',
 'diff_ptax_xdec',
 'dist1_abin',
 'dist1_adec',
 'dist1_xbin',
 'dist1_xdec',
 'dist2_abin',
 'dist2_adec',
 'dist2_xbin',
 'dist2_xdec'}

Yes. I’ll provide a mapping, but I won’t be able to do that until this evening.

Thanks @martinholmer

@hdoupe asked:

Would you mind giving me a mapping from the old results keys to the new results keys?
OLD:
'cdf_bin',
'cdf_dec',
'df_bin',
'df_dec',
'fiscal_tot_base',
'fiscal_tot_diffs',
'fiscal_tot_ref',
'mX_bin',
'mX_dec',
'mY_bin',
'mY_dec',
'pdf_bin',
'pdf_dec'

The above results table names are from TaxBrain not from the old 0.10.2 version of Tax-Calculator.
I don't know anything about that TaxBrain naming scheme.
What I can do is explain how the results table naming scheme in Tax-Calculator changed from 0.10.2 (which TaxBrain is now using) to 0.13.1.
Here is what was returned in in 0.10.2:
def dropq_summary(df1, df2, mask):
    """
    df1 contains raw results for the standard plan X and X'
    df2 contains raw results the user-specified plan (Plan Y)
    mask is the boolean mask where X and X' match
    """
    # pylint: disable=too-many-locals

    df1, df2 = drop_records(df1, df2, mask)

    # Totals for diff between baseline and reform
    dec_sum = (df2['tax_diff_dec'] * df2['s006']).sum()
    bin_sum = (df2['tax_diff_bin'] * df2['s006']).sum()
    pr_dec_sum = (df2['payrolltax_diff_dec'] * df2['s006']).sum()
    pr_bin_sum = (df2['payrolltax_diff_bin'] * df2['s006']).sum()
    combined_dec_sum = (df2['combined_diff_dec'] * df2['s006']).sum()
    combined_bin_sum = (df2['combined_diff_bin'] * df2['s006']).sum()

    # Totals for baseline
    sum_baseline = (df1['iitax'] * df1['s006']).sum()
    pr_sum_baseline = (df1['payrolltax'] * df1['s006']).sum()
    combined_sum_baseline = (df1['combined'] * df1['s006']).sum()

    # Totals for reform
    sum_reform = (df2['iitax_dec'] * df2['s006']).sum()
    pr_sum_reform = (df2['payrolltax_dec'] * df2['s006']).sum()
    combined_sum_reform = (df2['combined_dec'] * df2['s006']).sum()

    # Create difference tables, grouped by deciles and bins
    diffs_dec = dropq_diff_table(df1, df2,
                                 groupby='weighted_deciles',
                                 res_col='tax_diff',
                                 diff_col='iitax',
                                 suffix='_dec', wsum=dec_sum)

    diffs_bin = dropq_diff_table(df1, df2,
                                 groupby='webapp_income_bins',
                                 res_col='tax_diff',
                                 diff_col='iitax',
                                 suffix='_bin', wsum=bin_sum)

    pr_diffs_dec = dropq_diff_table(df1, df2,
                                    groupby='weighted_deciles',
                                    res_col='payrolltax_diff',
                                    diff_col='payrolltax',
                                    suffix='_dec', wsum=pr_dec_sum)

    pr_diffs_bin = dropq_diff_table(df1, df2,
                                    groupby='webapp_income_bins',
                                    res_col='payrolltax_diff',
                                    diff_col='payrolltax',
                                    suffix='_bin', wsum=pr_bin_sum)

    comb_diffs_dec = dropq_diff_table(df1, df2,
                                      groupby='weighted_deciles',
                                      res_col='combined_diff',
                                      diff_col='combined',
                                      suffix='_dec', wsum=combined_dec_sum)

    comb_diffs_bin = dropq_diff_table(df1, df2,
                                      groupby='webapp_income_bins',
                                      res_col='combined_diff',
                                      diff_col='combined',
                                      suffix='_bin', wsum=combined_bin_sum)

    m1_dec = create_distribution_table(df1, groupby='weighted_deciles',
                                       result_type='weighted_sum')

    m2_dec = dropq_dist_table(df2, groupby='weighted_deciles',
                              result_type='weighted_sum', suffix='_dec')

    m1_bin = create_distribution_table(df1, groupby='webapp_income_bins',
                                       result_type='weighted_sum')

    m2_bin = dropq_dist_table(df2, groupby='webapp_income_bins',
                              result_type='weighted_sum', suffix='_bin')

    return (m2_dec, m1_dec, diffs_dec, pr_diffs_dec, comb_diffs_dec,
            m2_bin, m1_bin, diffs_bin, pr_diffs_bin, comb_diffs_bin,
            dec_sum, pr_dec_sum, combined_dec_sum,
            sum_baseline, pr_sum_baseline, combined_sum_baseline,
            sum_reform, pr_sum_reform, combined_sum_reform)

And here is what is returned now:
def summary(df1, df2, mask):
    """
    df1 contains raw results for baseline plan
    df2 contains raw results for reform plan
    mask is the boolean array specifying records with reform-induced tax diffs
    returns dictionary of summary results DataFrames
    """
    # pylint: disable=too-many-statements,too-many-locals

    df2 = create_results_columns(df1, df2, mask)

    summ = dict()

    # tax difference totals between reform and baseline
    tdiff = df2['iitax_agg'] - df1['iitax']
    aggr_itax_d = (tdiff * df2['s006']).sum()
    tdiff = df2['payrolltax_agg'] - df1['payrolltax']
    aggr_ptax_d = (tdiff * df2['s006']).sum()
    tdiff = df2['combined_agg'] - df1['combined']
    aggr_comb_d = (tdiff * df2['s006']).sum()
    aggrd = [aggr_itax_d, aggr_ptax_d, aggr_comb_d]
    summ['aggr_d'] = pd.DataFrame(data=aggrd, index=AGGR_ROW_NAMES)

    # totals for baseline
    aggr_itax_1 = (df1['iitax'] * df1['s006']).sum()
    aggr_ptax_1 = (df1['payrolltax'] * df1['s006']).sum()
    aggr_comb_1 = (df1['combined'] * df1['s006']).sum()
    aggr1 = [aggr_itax_1, aggr_ptax_1, aggr_comb_1]
    summ['aggr_1'] = pd.DataFrame(data=aggr1, index=AGGR_ROW_NAMES)

    # totals for reform
    aggr_itax_2 = (df2['iitax_agg'] * df2['s006']).sum()
    aggr_ptax_2 = (df2['payrolltax_agg'] * df2['s006']).sum()
    aggr_comb_2 = (df2['combined_agg'] * df2['s006']).sum()
    aggr2 = [aggr_itax_2, aggr_ptax_2, aggr_comb_2]
    summ['aggr_2'] = pd.DataFrame(data=aggr2, index=AGGR_ROW_NAMES)

    # create difference tables grouped by xdec
    df2['iitax'] = df2['iitax_xdec']
    summ['diff_itax_xdec'] = \
        create_difference_table(df1, df2,
                                groupby='weighted_deciles',
                                income_measure='expanded_income',
                                tax_to_diff='iitax')

    df2['payrolltax'] = df2['payrolltax_xdec']
    summ['diff_ptax_xdec'] = \
        create_difference_table(df1, df2,
                                groupby='weighted_deciles',
                                income_measure='expanded_income',
                                tax_to_diff='payrolltax')

    df2['combined'] = df2['combined_xdec']
    summ['diff_comb_xdec'] = \
        create_difference_table(df1, df2,
                                groupby='weighted_deciles',
                                income_measure='expanded_income',
                                tax_to_diff='combined')

    # create difference tables grouped by xbin (removing negative-income bin)
    df2['iitax'] = df2['iitax_xbin']
    diff_itax_xbin = \
        create_difference_table(df1, df2,
                                groupby='webapp_income_bins',
                                income_measure='expanded_income',
                                tax_to_diff='iitax')
    diff_itax_xbin.drop(diff_itax_xbin.index[0], inplace=True)
    summ['diff_itax_xbin'] = diff_itax_xbin

    df2['payrolltax'] = df2['payrolltax_xbin']
    diff_ptax_xbin = \
        create_difference_table(df1, df2,
                                groupby='webapp_income_bins',
                                income_measure='expanded_income',
                                tax_to_diff='payrolltax')
    diff_ptax_xbin.drop(diff_ptax_xbin.index[0], inplace=True)
    summ['diff_ptax_xbin'] = diff_ptax_xbin

    df2['combined'] = df2['combined_xbin']
    diff_comb_xbin = \
        create_difference_table(df1, df2,
                                groupby='webapp_income_bins',
                                income_measure='expanded_income',
                                tax_to_diff='combined')
    diff_comb_xbin.drop(diff_comb_xbin.index[0], inplace=True)
    summ['diff_comb_xbin'] = diff_comb_xbin

    # create distribution tables grouped by xdec
    summ['dist1_xdec'] = \
        create_distribution_table(df1, groupby='weighted_deciles',
                                  income_measure='expanded_income',
                                  result_type='weighted_sum')

    suffix = '_xdec'
    df2_cols_with_suffix = [c for c in list(df2) if c.endswith(suffix)]
    for col in df2_cols_with_suffix:
        root_col_name = col.replace(suffix, '')
        df2[root_col_name] = df2[col]
    df2['expanded_income_baseline'] = df1['expanded_income']
    summ['dist2_xdec'] = \
        create_distribution_table(df2, groupby='weighted_deciles',
                                  income_measure='expanded_income_baseline',
                                  result_type='weighted_sum')

    # create distribution tables grouped by xbin (removing negative-income bin)
    dist1_xbin = \
        create_distribution_table(df1, groupby='webapp_income_bins',
                                  income_measure='expanded_income',
                                  result_type='weighted_sum')
    dist1_xbin.drop(dist1_xbin.index[0], inplace=True)
    summ['dist1_xbin'] = dist1_xbin

    suffix = '_xbin'
    df2_cols_with_suffix = [c for c in list(df2) if c.endswith(suffix)]
    for col in df2_cols_with_suffix:
        root_col_name = col.replace(suffix, '')
        df2[root_col_name] = df2[col]
    df2['expanded_income_baseline'] = df1['expanded_income']
    dist2_xbin = \
        create_distribution_table(df2, groupby='webapp_income_bins',
                                  income_measure='expanded_income_baseline',
                                  result_type='weighted_sum')
    dist2_xbin.drop(dist2_xbin.index[0], inplace=True)
    summ['dist2_xbin'] = dist2_xbin

    # create difference tables grouped by adec
    df2['iitax'] = df2['iitax_adec']
    summ['diff_itax_adec'] = \
        create_difference_table(df1, df2,
                                groupby='weighted_deciles',
                                income_measure='c00100',
                                tax_to_diff='iitax')

    df2['payrolltax'] = df2['payrolltax_adec']
    summ['diff_ptax_adec'] = \
        create_difference_table(df1, df2,
                                groupby='weighted_deciles',
                                income_measure='c00100',
                                tax_to_diff='payrolltax')

    df2['combined'] = df2['combined_adec']
    summ['diff_comb_adec'] = \
        create_difference_table(df1, df2,
                                groupby='weighted_deciles',
                                income_measure='c00100',
                                tax_to_diff='combined')

    # create difference tables grouped by abin (removing negative-income bin)
    df2['iitax'] = df2['iitax_abin']
    diff_itax_abin = \
        create_difference_table(df1, df2,
                                groupby='webapp_income_bins',
                                income_measure='c00100',
                                tax_to_diff='iitax')
    diff_itax_abin.drop(diff_itax_abin.index[0], inplace=True)
    summ['diff_itax_abin'] = diff_itax_abin

    df2['payrolltax'] = df2['payrolltax_abin']
    diff_ptax_abin = \
        create_difference_table(df1, df2,
                                groupby='webapp_income_bins',
                                income_measure='c00100',
                                tax_to_diff='payrolltax')
    diff_ptax_abin.drop(diff_ptax_abin.index[0], inplace=True)
    summ['diff_ptax_abin'] = diff_ptax_abin

    df2['combined'] = df2['combined_abin']
    diff_comb_abin = \
        create_difference_table(df1, df2,
                                groupby='webapp_income_bins',
                                income_measure='c00100',
                                tax_to_diff='combined')
    diff_comb_abin.drop(diff_comb_abin.index[0], inplace=True)
    summ['diff_comb_abin'] = diff_comb_abin

    # create distribution tables grouped by adec
    summ['dist1_adec'] = \
        create_distribution_table(df1, groupby='weighted_deciles',
                                  income_measure='c00100',
                                  result_type='weighted_sum')

    suffix = '_adec'
    df2_cols_with_suffix = [c for c in list(df2) if c.endswith(suffix)]
    for col in df2_cols_with_suffix:
        root_col_name = col.replace(suffix, '')
        df2[root_col_name] = df2[col]
    df2['c00100_baseline'] = df1['c00100']
    summ['dist2_adec'] = \
        create_distribution_table(df2, groupby='weighted_deciles',
                                  income_measure='c00100_baseline',
                                  result_type='weighted_sum')

    # create distribution tables grouped by abin (removing negative-income bin)
    dist1_abin = \
        create_distribution_table(df1, groupby='webapp_income_bins',
                                  income_measure='c00100',
                                  result_type='weighted_sum')
    dist1_abin.drop(dist1_abin.index[0], inplace=True)
    summ['dist1_abin'] = dist1_abin

    suffix = '_abin'
    df2_cols_with_suffix = [c for c in list(df2) if c.endswith(suffix)]
    for col in df2_cols_with_suffix:
        root_col_name = col.replace(suffix, '')
        df2[root_col_name] = df2[col]
    df2['c00100_baseline'] = df1['c00100']
    dist2_abin = \
        create_distribution_table(df2, groupby='webapp_income_bins',
                                  income_measure='c00100_baseline',
                                  result_type='weighted_sum')
    dist2_abin.drop(dist2_abin.index[0], inplace=True)
    summ['dist2_abin'] = dist2_abin

    # return dictionary of summary results
    return summ

So, you can see that now there is a consistent naming scheme and many more tables (because we now produce deciles and bins for AGI as well as for expanded income).
The new results table names are:
----------------------------------------------------------
0.13.1                        0.10.2
----------------------------------------------------------
aggr_1                        were separate by tax type
aggr_2                        were separate by tax type
aggr_d                        were separate by tax type
----------------------------------------------------------
dist1_abin                    ---
dist1_adec                    ---
dist1_xbin                    m1_bin
dist1_xdec                    m1_dec
dist2_abin                    ---
dist2_adec                    ---
dist2_xbin                    m2_bin
dist2_xdec                    m2_dec
----------------------------------------------------------
diff_comb_abin                ---
diff_comb_adec                ---
diff_comb_xbin                comb_diffs_bin
diff_comb_xdec                comb_diffs_dec
diff_itax_abin                ---
diff_itax_adec                ---
diff_itax_xbin                diffs_bin
diff_itax_xdec                diffs_dec
diff_ptax_abin                ---
diff_ptax_adec                ---
diff_ptax_xbin                pr_diffs_bin
diff_ptax_xdec                pr_diffs_dec
----------------------------------------------------------

As before, there are three sets of tables: aggregate (aggr), distribution (dist), and difference {diff) tables.
1 stands for baseline and 2 stands for reform and d for difference (only for the aggregate tables).
comb stands for combined, itax standard for income tax, and ptax stands for payroll tax.
xdec stands for expanded income deciles
adec stands for AGI deciles
xbin stands for expanded income bins
abin stands for AGI bins
Let me know if you have any questions.

Thanks @martinholmer, that's exactly what I needed.

PR #738 is nearing completion and is ready to begin being reviewed.
Places to review:

tax_result decorator as described by @talumbau was implemented. @talumbau if you have time to review this then that would be greatly appreciated.  Most of this was done in this commit and this commit
User input processing was scattered between */views.py, */compute.py, flask_server.py and celery_tasks.py.  I tried to move all of this into the */views.py files.
The package_up_user_vars() tests were removed in this commit.  They depended heavily on the inflation rates which were updated in open-source-economics/taxdata#121.  Since B-Tax is the only app that still depends on this function, I think we are safe in removing these tests instead of updating them.  However, this is up for debate.
The tests used to set the environment variable, NUM_BUDGET_YEARS to 2.  I had trouble with this in this PR and have had trouble with this in PR #676.  There is an issue with when this variable is changed and when imported modules such as taxbrain/compute.py set module level variables to NUM_BUDGET_YEARS.  There appears to be a discrepancy between when the test modules set a module variable NUM_BUDGET_YEARS and when the regular modules like taxbrain/compute.py set a module variable NUM_BUDGET_YEARS.  e.g. taxbrain/compute.py vs  taxbrain/views.py  In this commit, I removed this behavior due to the seeming fragility (unless I've broken something which is possible) and to not seeing how it is particularly useful.  @talumbau if you have input on this then that would also be appreciated.
@Amy-Xu @martinholmer Just to verify, the gdp elasticity model skips year 0 (the first year), correct?

@brittainhard if you have time to review that would be very helpful.
TODO:

[ ] 1. Iron out final table issues.
[ ] 2. General clean up

@hdoupe said:

Just to verify, the gdp elasticity model skips year 0 (the first year), correct?

No, all years should have numerical values. Look at the new Tax-Calculator code.

@martinholmer I'm getting the following error when I try to run with year_n equal to 0:
[2017-11-17 13:42:57,204: WARNING/MainProcess] {'start_year': 2017, 'use_puf_not_cps': True, 'use_full_sample': True, 'year_n': 0, 'gdp_elasticity': 0.3, 'user_mods': {u'policy': {2017: {u'_ALD_InvInc_ec_base_RyanBrady': [False], u'_NIIT_PT_taxed': [False], u'_FICA_ss_trt': [0.1], u'_ID_BenefitCap_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_PT_wages_active_income': [False], u'_EITC_indiv': [False], u'_ID_BenefitSurtax_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_CTC_new_for_all': [False], u'_PT_top_stacking': [True], u'_ID_AmountCap_Switch': [[1, 1, 1, 1, 1, 1, True]], u'_II_no_em_nu18': [False], u'_CG_nodiff': [False], u'_CTC_new_refund_limited': [False]}}, u'growdiff_response': {}, u'behavior': {}, u'consumption': {}, u'growdiff_baseline': {}}}
[2017-11-17 13:42:59,854: WARNING/MainProcess] puf-read-time= 2.5
[2017-11-17 13:43:01,147: WARNING/MainProcess] You loaded data for 2009.
[2017-11-17 13:43:01,230: WARNING/MainProcess] Tax-Calculator startup automatically extrapolated your data to 2013.
[2017-11-17 13:43:03,070: WARNING/MainProcess] You loaded data for 2009.
[2017-11-17 13:43:03,153: WARNING/MainProcess] Tax-Calculator startup automatically extrapolated your data to 2013.
[2017-11-17 13:43:05,244: WARNING/MainProcess] You loaded data for 2009.
[2017-11-17 13:43:05,377: WARNING/MainProcess] Tax-Calculator startup automatically extrapolated your data to 2013.
[2017-11-17 13:43:07,427: ERROR/MainProcess] Task celery_tasks.elasticity_gdp_task_async[bbe9e42d-0738-4431-974e-aa1cc0afcb6e] raised unexpected: ValueError('calc.current_year=2017 must be one less than year=2017',)
Traceback (most recent call last):
  File "/anaconda/envs/aei_dropq/lib/python2.7/site-packages/celery/app/trace.py", line 374, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/anaconda/envs/aei_dropq/lib/python2.7/site-packages/celery/app/trace.py", line 629, in __protected_call__
    return self.run(*args, **kwargs)
  File "/Users/henrydoupe/Documents/PolicyBrain/deploy/taxbrain_server/celery_tasks.py", line 124, in elasticity_gdp_task_async
    gdp_elasticity=gdp_elasticity
  File "/anaconda/envs/aei_dropq/lib/python2.7/site-packages/taxcalc/tbi/tbi.py", line 189, in run_nth_year_gdp_elast_model
    calc1, calc2, gdp_elasticity)
  File "/anaconda/envs/aei_dropq/lib/python2.7/site-packages/taxcalc/macro_elasticity.py", line 70, in proportional_change_in_gdp
    raise ValueError(msg.format(calc1.current_year, year))
ValueError: calc.current_year=2017 must be one less than year=2017
[2017-11-17 13:43:07,546: INFO/MainProcess] Received task: celery_tasks.elasticity_gdp_task_async[fef11755-49c7-4422-bd0c-2e183b571104]
[2017-11-17 13:43:07,548: WARNING/MainProcess] kw to dropq
[2017-11-17 13:43:07,548: WARNING/MainProcess] {'start_year': 2017, 'use_puf_not_cps': True, 'use_full_sample': True, 'year_n': 1, 'gdp_elasticity': 0.3, 'user_mods': {u'policy': {2017: {u'_ALD_InvInc_ec_base_RyanBrady': [False], u'_NIIT_PT_taxed': [False], u'_FICA_ss_trt': [0.1], u'_ID_BenefitCap_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_PT_wages_active_income': [False], u'_EITC_indiv': [False], u'_ID_BenefitSurtax_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_CTC_new_for_all': [False], u'_PT_top_stacking': [True], u'_ID_AmountCap_Switch': [[1, 1, 1, 1, 1, 1, True]], u'_II_no_em_nu18': [False], u'_CG_nodiff': [False], u'_CTC_new_refund_limited': [False]}}, u'growdiff_response': {}, u'behavior': {}, u'consumption': {}, u'growdiff_baseline': {}}}
[2017-11-17 13:43:10,087: WARNING/MainProcess] puf-read-time= 2.5
[2017-11-17 13:43:11,429: WARNING/MainProcess] You loaded data for 2009.
[2017-11-17 13:43:11,533: WARNING/MainProcess] Tax-Calculator startup automatically extrapolated your data to 2013.
[2017-11-17 13:43:13,353: WARNING/MainProcess] You loaded data for 2009.
[2017-11-17 13:43:13,448: WARNING/MainProcess] Tax-Calculator startup automatically extrapolated your data to 2013.
[2017-11-17 13:43:15,438: WARNING/MainProcess] You loaded data for 2009.
[2017-11-17 13:43:15,537: WARNING/MainProcess] Tax-Calculator startup automatically extrapolated your data to 2013.
[2017-11-17 13:43:20,035: INFO/MainProcess] Task celery_tasks.elasticity_gdp_task_async[fef11755-49c7-4422-bd0c-2e183b571104] succeeded in 12.486489125s: '{"elasticity_gdp": {"gdp_proportional_change_1": "0.00640"}, "dropq_version": "0.13.0", "taxcalc_version": "0.13.0"}'
[2017-11-17 13:43:20,039: INFO/MainProcess] Received task: celery_tasks.elasticity_gdp_task_async[4916db39-b8ae-475d-8d35-395421ca17d3]

Here is code to recreate this error:
import taxcalc
# year_n = 0; throws error
keywords = {'start_year': 2017, 'use_puf_not_cps': True, 'use_full_sample': True, 'year_n': 0, 'gdp_elasticity': 0.3, 'user_mods': {u'policy': {2017: {u'_ALD_InvInc_ec_base_RyanBrady': [False], u'_NIIT_PT_taxed': [False], u'_FICA_ss_trt': [0.1], u'_ID_BenefitCap_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_PT_wages_active_income': [False], u'_EITC_indiv': [False], u'_ID_BenefitSurtax_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_CTC_new_for_all': [False], u'_PT_top_stacking': [True], u'_ID_AmountCap_Switch': [[1, 1, 1, 1, 1, 1, True]], u'_II_no_em_nu18': [False], u'_CG_nodiff': [False], u'_CTC_new_refund_limited': [False]}}, u'growdiff_response': {}, u'behavior': {}, u'consumption': {}, u'growdiff_baseline': {}}}
keywords['use_puf_not_cps'] = False
res = taxcalc.tbi.run_nth_year_gdp_elast_model(**keywords)

# year_n = 1; doesn't throw error
keywords = {'start_year': 2017, 'use_puf_not_cps': True, 'use_full_sample': True, 'year_n': 1, 'gdp_elasticity': 0.3, 'user_mods': {u'policy': {2017: {u'_ALD_InvInc_ec_base_RyanBrady': [False], u'_NIIT_PT_taxed': [False], u'_FICA_ss_trt': [0.1], u'_ID_BenefitCap_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_PT_wages_active_income': [False], u'_EITC_indiv': [False], u'_ID_BenefitSurtax_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_CTC_new_for_all': [False], u'_PT_top_stacking': [True], u'_ID_AmountCap_Switch': [[1, 1, 1, 1, 1, 1, True]], u'_II_no_em_nu18': [False], u'_CG_nodiff': [False], u'_CTC_new_refund_limited': [False]}}, u'growdiff_response': {}, u'behavior': {}, u'consumption': {}, u'growdiff_baseline': {}}}
keywords['use_puf_not_cps'] = False
res = taxcalc.tbi.run_nth_year_gdp_elast_model(**keywords)

I'm having trouble figuring out what is broken here.
cc @Amy-Xu

@hdoupe provided this code snippet:
import taxcalc
# year_n = 0; throws error
keywords = {'start_year': 2017, 'use_puf_not_cps': True, 'use_full_sample': True, 'year_n': 0, 'gdp_elasticity': 0.3, 'user_mods': {u'policy': {2017: {u'_ALD_InvInc_ec_base_RyanBrady': [False], u'_NIIT_PT_taxed': [False], u'_FICA_ss_trt': [0.1], u'_ID_BenefitCap_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_PT_wages_active_income': [False], u'_EITC_indiv': [False], u'_ID_BenefitSurtax_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_CTC_new_for_all': [False], u'_PT_top_stacking': [True], u'_ID_AmountCap_Switch': [[1, 1, 1, 1, 1, 1, True]], u'_II_no_em_nu18': [False], u'_CG_nodiff': [False], u'_CTC_new_refund_limited': [False]}}, u'growdiff_response': {}, u'behavior': {}, u'consumption': {}, u'growdiff_baseline': {}}}
keywords['use_puf_not_cps'] = False
res = taxcalc.tbi.run_nth_year_gdp_elast_model(**keywords)

# year_n = 1; doesn't throw error
keywords = {'start_year': 2017, 'use_puf_not_cps': True, 'use_full_sample': True, 'year_n': 1, 'gdp_elasticity': 0.3, 'user_mods': {u'policy': {2017: {u'_ALD_InvInc_ec_base_RyanBrady': [False], u'_NIIT_PT_taxed': [False], u'_FICA_ss_trt': [0.1], u'_ID_BenefitCap_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_PT_wages_active_income': [False], u'_EITC_indiv': [False], u'_ID_BenefitSurtax_Switch': [[1, 1, 1, 1, 1, 1, 1]], u'_CTC_new_for_all': [False], u'_PT_top_stacking': [True], u'_ID_AmountCap_Switch': [[1, 1, 1, 1, 1, 1, True]], u'_II_no_em_nu18': [False], u'_CG_nodiff': [False], u'_CTC_new_refund_limited': [False]}}, u'growdiff_response': {}, u'behavior': {}, u'consumption': {}, u'growdiff_baseline': {}}}
keywords['use_puf_not_cps'] = False
res = taxcalc.tbi.run_nth_year_gdp_elast_model(**keywords)

Is it true (or not) that in TaxBrain the start_year (in the code above) is the first year in which any reform provisions can be implemented?

@martinholmer asked

Is it true (or not) that in TaxBrain the start_year (in the code above) is the first year in which any reform provisions can be implemented?

Yes, from the GUI, if you select the start year to be 2017, you can set values in the future using the * placeholder but you cannot set values into the past.

@GoFroggyRun I'm still having trouble getting the tables to show up in TaxBrain.  Do you know where in the javascript these tables are created?

@hdoupe the JS portion of those tables should be in taxbrain-tablebuilder.js or similar files.
When I checkout #738 locally and try to run local page, I wasn't able to do so. Some error message reads:
AttributeError: 'module' object has no attribute 'DIST_TABLE_LABELS'

@GoFroggyRun which Tax-Calculator version are you running?

@hdoupe it's still on 0.10.2 even if I manually update TC. Any ideas?

First, check conda-requirements.txt to make sure that you have taxcalc==0.13.0
Then I would try resetting the environment with
cd PolicyBrain/  && source deactivate && conda env remove --name aei_dropq && pushd deploy/ && ./install_taxbrain_server.sh && popd && source activate aei_dropq && source webapp_env.sh

@GoFroggyRun I've been going through the taxbrain-tablebuilder.js script.  I'm having trouble figuring out where it pulls the table names from the tables key in the context.
My question is what happens when I click one of these buttons:

The table has keys:
 dict_keys(['result_years', 'tooltips', 'dist1_xbin', 'fiscal_currentlaw', 'diff_ptax_xdec', 'fiscal_change', 'diff_comb_xbin', 'dist2_xdec', 'dist1_xdec', 'diff_comb_xdec', 'diff_ptax_xbin', 'dist2_xbin', 'diff_itax_xdec', 'diff_itax_xbin', 'fiscal_reform'])

How does the javascipt know the name of the table that it wants?  How does it know to pull the diff_itax_xbin label from this dictionary when the user clicks Income Tax?  I think our problem can be solved if we update this mapping.

I think our problem can be solved if we update this mapping.

I can confirm that this is the case.  I swapped one of the new names back to the old name and that table loaded while the other tables still did not.

@martinholmer PR open-source-economics/Tax-Calculator#1677 resolved the GDP elasticity table issue.  Thanks for the good work.

@hdoupe asked:

How does the javascipt know the name of the table that it wants? How does it know to pull the diff_itax_xbin label from this dictionary when the user clicks Income Tax? I think our problem can be solved if we update this mapping.

I believe this is handled here in the taxbrain-tablebuilder.js file. The event section corresponds to the manipulations you described.

The most recent commit fixes the problem with the tables not showing up.  This is a temporary solution that needs to be replaced with a real solution.  However, I think this involves having some knowledge of java script and front-end engineering, which I do not have.

@GoFroggyRun thanks for finding that.  But, I'm unable to find the place where it selects the table JSON object and then selects a key such as mY_dec under the old naming scheme or dist2_xdec under the new naming scheme.  If we can find that place and swap out the table names, then this would be a better solution than what I did in this commit.

The only remaining problem (that I know of) is that the sections containing new parameter names are not collapsed when you go to the input page with all default parameters.
@GoFroggyRun Do you know where the collapse/expand logic is and how to fix that?
I would say that this is a more pressing issue than the table mapping issue, given the time constraint and that we have a working solution to the table mapping issue.

@hdoupe said:

But, I'm unable to find the place where it selects the table JSON object and then selects a key such as mY_dec under the old naming scheme or dist2_xdec under the new naming scheme.

Does this look like something helpful?

Do you know where the collapse/expand logic is and how to fix that?

Not really. Never worked on it before. I will checkout your PR and take a look.

@GoFroggyRun that was updated with the new table names in this commit

@GoFroggyRun said

Not really. Never worked on it before. I will checkout your PR and take a look.

Great, thanks.

@hayleefay do you have any ideas on how to solve either of these front-end issues (the table mapping isssue and the collapse/expand issue)?

@brittainhard do you have a minute to take a look at the collapse/expand bug that we are trying to solve here?
cc @atxsder


Fixes #410, adds note about minimum Go version

I was doing some regression testing and noticed that development requires Go 1.5+ now, so I added a note to the CONTRIBUTING.md file

Fixed already.
BTW could you start you commit messages with a lowercase verb such as "add", "fix", "refactor", "remove" :) We have that mentioned in contribution doc.

I have a pre hook that capitalizes the first word. I'll disable it on the Apex repo. Didn't even realize it was happening, TBH! Thanks for the suggestion, @mthenw


Could you please support stock exchange?

thanks for all of you
i notice that this application just for Forex exchange
could you please support stock exchange?

Thanks for feedback. Good question.
Firstly the application supports OANDA broker, hence Forex market.
The main use case of the application is providing a platform to play with trading strategies, not creating a user interface for a user. Bottom line the charts part is useless. at least for me.
The application cannot compete with other professional offer. Just the default application provided by OANDA is very good in terms of charts and graph.
Due the focus on strategies, I need a market opened 24H and very liquid.
Furthermore supporting the stock exchange market, you need to find a free broker with no delay and also with historical prices. Then you need to create the adapter for that broker and integrate it in the application. I suppose you know the kind of the effort.
Feel free to offer a proposal and, finally, to contribute.

Got it
thanks for your reply :)


Finish animation when play is changed from true to false.

Currently, when play is changed from true to false, the animation just pauses.
Is there a way for it to just finish and then stop?

I found a way that involves segments, loop, and play.
Closing the issue.

@Ey-Jay mind sharing?

@Ey-Jay mind sharing?


Scroll within a section

For the Different Heights feature, when the section is much taller than the viewport height, scrolling down will take you to the next section, cutting off the rest of the text of the current section that lies 'below' the viewport.
It would be an improvement / feature to allow normal scrolling until you reach the bottom of the current section, and then section scrolling will take effect again.

Resolved in ea24834


[#441] check for pipe at startup of forwarding trace

description

[x] check for existence of pipe before starting up a trace forwarding backend. This should prevent the case where the node is started before the trace acceptor, which creates the pipe. In that case the node will create a plain file and start writing traced objects to it without log rotation active on this file. That file might grow to very larg size.

checklist

[x] compiles (cabal new-clean; cabal new-build)
[x] tests run successfully (cabal new-test)
[ ] documentation added and created (cd docs; nix-shell --run make)
[x] link to an issue
[ ] link to an epic
[ ] add estimate points
[x] add milestone (the same as the linked issue)

I was surprised that WriteMode seems to imply C's O_CREAT.


Add visualization callback for solvers

Callback could called to update visualizations;
ps: check how channels would work for it

Done; Channels work great for this use-case; Issue was how to share rendering channels between solvers;


Demo app returns always 500 internal server error

PayPal Android SDK Version
0.0.13
Environment
Sandbox
Android Version & Device
Android 13, Google Pixel 7
PayPal dependencies
Demo app "as is"
Describe the bug
I have checked out the SDK and wanted to try out the Demo app. However, doesn't matter what option I choose, it always ends up showing the "500 Internal server error" when trying to execute a call.
To reproduce

Start Demo app
Open "Paypal native"
Click "Create order and continue"
In PaypalNativeFragment, click "Get client ID"
Client ID fetched is shown, along with shipping options
Select "No shipping"
Click "Start checkout"
A "500 internal server error" is logged as follows:

2023-08-29 17:07:57.985 --> POST https://sdk-sample-merchant-server.herokuapp.com/orders
2023-08-29 17:07:57.985 Content-Type: application/json; charset=UTF-8
2023-08-29 17:07:57.985 Content-Length: 1376
2023-08-29 17:07:57.986 {"application_context":{"brand_name":"EXAMPLE INC","cancel_url":"https://example.com/cancel","landing_page":"BILLING","locale":"de-DE","return_url":"https://example.com/return","shipping_preference":"NO_SHIPPING","user_action":"CONTINUE"},"intent":"AUTHORIZE","purchase_units":[{"amount":{"breakdown":{"discount":{"currency_code":"USD","value":"00.00"},"handling":{"currency_code":"USD","value":"00.00"},"item_total":{"currency_code":"USD","value":"5.0"},"shipping":{"currency_code":"USD","value":"0,00"},"shipping_discount":{"currency_code":"USD","value":"00.00"},"tax_total":{"currency_code":"USD","value":"00.00"}},"currency_code":"USD","value":"5,00"},"custom_id":"CUST-HighFashions","description":"Sporting Goods","items":[{"category":"PHYSICAL_GOODS","description":"Green XL","name":"T-Shirt","quantity":"1","sku":"sku01","tax":{"currency_code":"USD","value":"00.00"},"unit_amount":{"currency_code":"USD","value":"00.00"}},{"category":"PHYSICAL_GOODS","description":"Running, Size 10.5","name":"Shoes","quantity":"2","sku":"sku02","tax":{"currency_code":"USD","value":"00.00"},"unit_amount":{"currency_code":"USD","value":"00.00"}}],"reference_id":"PUHF","shipping":{"address":{"address_line_1":"123 Townsend St","address_line_2":"Floor 6","admin_area_1":"CA","admin_area_2":"San Francisco","country_code":"US","postal_code":"94107"}},"soft_descriptor":"HighFashions"}]}
2023-08-29 17:07:57.986 --> END POST (1376-byte body)
2023-08-29 17:07:59.494 <-- 500 Internal Server Error https://sdk-sample-merchant-server.herokuapp.com/orders (1507ms)
2023-08-29 17:07:59.495 Server: Cowboy
2023-08-29 17:07:59.495 Connection: keep-alive
2023-08-29 17:07:59.495 X-Powered-By: Express
2023-08-29 17:07:59.495 Access-Control-Allow-Origin: *
2023-08-29 17:07:59.495 Access-Control-Allow-Credentials: true
2023-08-29 17:07:59.496 Access-Control-Allow-Methods: GET,PUT,POST,DELETE,OPTIONS,PATCH
2023-08-29 17:07:59.496 Access-Control-Allow-Headers: Origin,X-Requested-With,Content-Type,Accept,content-type,application/json
2023-08-29 17:07:59.496 Content-Type: application/json; charset=utf-8
2023-08-29 17:07:59.496 Content-Length: 2410
2023-08-29 17:07:59.496 Etag: W/"96a-OWJsOoDFE0lYpnRYifqbiFKkq+0"
2023-08-29 17:07:59.496 Date: Tue, 29 Aug 2023 15:07:57 GMT
2023-08-29 17:07:59.496 Via: 1.1 vegur
2023-08-29 17:07:59.498 {"internal":{"message":"Request failed with status code 400","name":"AxiosError","config":{"transitional":{"silentJSONParsing":true,"forcedJSONParsing":true,"clarifyTimeoutError":false},"transformRequest":[null],"transformResponse":[null],"timeout":0,"xsrfCookieName":"XSRF-TOKEN","xsrfHeaderName":"X-XSRF-TOKEN","maxContentLength":-1,"maxBodyLength":-1,"env":{},"headers":{"Accept":"application/json, text/plain, */*","Content-Type":"application/json","Authorization":"Bearer A21AAIH6wxcodgajxCCV_CTuRFc2nmpy773TWnApf9lIhlzWt3mLV1wTPu8PKmfTURi-_BiqMhRZhg_StPO4dAK2o42pSixVw","paypal-request-id":"1693321677297","User-Agent":"axios/0.27.2","Content-Length":1376},"baseURL":"https://api.sandbox.paypal.com","method":"post","url":"/v2/checkout/orders","data":"{\"application_context\":{\"brand_name\":\"EXAMPLE INC\",\"cancel_url\":\"https://example.com/cancel\",\"landing_page\":\"BILLING\",\"locale\":\"de-DE\",\"return_url\":\"https://example.com/return\",\"shipping_preference\":\"NO_SHIPPING\",\"user_action\":\"CONTINUE\"},\"intent\":\"AUTHORIZE\",\"purchase_units\":[{\"amount\":{\"breakdown\":{\"discount\":{\"currency_code\":\"USD\",\"value\":\"00.00\"},\"handling\":{\"currency_code\":\"USD\",\"value\":\"00.00\"},\"item_total\":{\"currency_code\":\"USD\",\"value\":\"5.0\"},\"shipping\":{\"currency_code\":\"USD\",\"value\":\"0,00\"},\"shipping_discount\":{\"currency_code\":\"USD\",\"value\":\"00.00\"},\"tax_total\":{\"currency_code\":\"USD\",\"value\":\"00.00\"}},\"currency_code\":\"USD\",\"value\":\"5,00\"},\"custom_id\":\"CUST-HighFashions\",\"description\":\"Sporting Goods\",\"items\":[{\"category\":\"PHYSICAL_GOODS\",\"description\":\"Green XL\",\"name\":\"T-Shirt\",\"quantity\":\"1\",\"sku\":\"sku01\",\"tax\":{\"currency_code\":\"USD\",\"value\":\"00.00\"},\"unit_amount\":{\"currency_code\":\"USD\",\"value\":\"00.00\"}},{\"category\":\"PHYSICAL_GOODS\",\"description\":\"Running, Size 10.5\",\"name\":\"Shoes\",\"quantity\":\"2\",\"sku\":\"sku02\",\"tax\":{\"currency_code\":\"USD\",\"value\":\"00.00\"},\"unit_amount\":{\"currency_code\":\"USD\",\"value\":\"00.00\"}}],\"reference_id\":\"PUHF\",\"shipping\":{\"address\":{\"address_line_1\":\"123 Townsend St\",\"address_line_2\":\"Floor 6\",\"admin_area_1\":\"CA\",\"admin_area_2\":\"San Francisco\",\"country_code\":\"US\",\"postal_code\":\"94107\"}},\"soft_descriptor\":\"HighFashions\"}]}"},"code":"ERR_BAD_REQUEST","status":400}}
2023-08-29 17:07:59.498 <-- END HTTP (2410-byte body)

Expected behavior
The Demo app works out of the box or some guidance is provided, how to set it up.
Screenshots
No response

👋 Hi @lukaszkalnik - thanks for using the PayPal Android SDK and for opening an issue with us.
On the latest main branch (running on a Samsung Galaxy S22 device) I am able to use the Demo app just fine. Maybe you have some sort of firewall blocking our herokuapp.com sample server?


Nightly demo produces NO_VIDEO_ELEMENT after a failed load

Have you read the FAQ and checked for duplicate open issues?
Yes
What version of Shaka Player are you using?
master
Can you reproduce the issue with our latest release version?
no - does not affect v2.4.7 or v2.5.0-beta3
Can you reproduce the issue with the latest code from master?
yes
Are you using the demo app or your own custom app?
demo app
What browser and OS are you using?
ChromeOS 72, but it happens on desktop Chrome, too
What are the manifest and license server URIs?
Any that produce an error during loading.
What did you do?

Load anything that will fail to load, such as the URL "foo": https://nightly-dot-shaka-player-demo.appspot.com/demo/#asset=foo;lang=de;build=uncompiled
Then click load again

What did you expect to happen?
The same NETWORK.BAD_HTTP_STATUS (https:foo,404,,[object Object],0) error both times, as happens on v2.5.0-beta3 and v2.4.7.
What actually happened?

The second load results in the error PLAYER.NO_VIDEO_ELEMENT ().  It seems that something is wrong with the latest load graph changes.
Changing to another asset that should work still results in NO_VIDEO_ELEMENT.  So it seems that the first error put the Player into an unrecoverable error state.

This issue is on my radar, there is a TODO in the handleError method in shaka.Player with all the information needed to fix the issue.
Because I know that reloading the page each time an error occurs can interrupt one's workflow, I can't image we would want this to issue to remain open for longer than necessary. Since I am out of the office until Monday, I would invite anyone interested in helping out to read the information in the TODO and resolve the issue.

Awesome, thanks @vaage!

I can confirm that I am also seeing this on Tizen as well, possibly after creating additional player instances.


stie gici business school

@firmansari First stage of verification passed, official domain is stiegici.ac.id as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@firmansari Please provide us with an URL of a page on the official website where the school offers an IT-related long-term (>1 year) course. Thank you.

Here is our campus url https://stiegici.ac.id/
we also provide online learning at https://elearning.stiegici.ac.id/ and there are several IT-related courses

@firmansari  Thank you for the clarifications. Pull request merged.


More strict types of callback parameters

Types of lifecycle callback parameters are very generic, namely object. We could leverage polymorphic this and mapped (Partial in particular) types to make it more strict.
I'll follow up with a PR.

Sorry for the delay on this. The API has changed a good bit (merged to master and to be released in version 0.6). Please re-file this issue if the problem persists in the new version. Thanks.


[Search v2.2] [App] Refactor Transaction and Report types to Expense type

Implement this section of the design doc
Issue OwnerCurrent Issue Owner: @luacmartins

Still waiting on doc reviews.

Implementation on-going

Remind me, are you paid via Upwork or Expensify?

Offer sent

All paid out! Closing this one, as regression and related considerations will be handled centrally.


implementation of #2702

對應到#2702 的改做.
加入了JDBCSchemaObject類和SchemaRepositry 的 registerSchemaObject 方法

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Admin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.Admin seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

代码距离现在太久了，不方便合并，因此关掉。
有需要的请重新提交新的pr


Query on tests / validation for conda GNU compilers for new architecture.

Hello,
I am trying to understand - what all tests and validation are recommended to bring up conda compiler for new architecture.
There are test stage in meta.yaml for gcc,g++ , others in ctng-compilers-feedstock and below test coverage in build.sh.
https://github.com/conda-forge/ctng-compilers-feedstock/blob/883e60a8f7b1b3d7b16e794605ee61a9541943a0/recipe/build.sh#L112
# pushd .build/${CHOST}/build/build-cc-gcc-final
# make -k check || true
# popd

are above tests strictly recommended for new architecture ? Any suggestion would really help.
Thank you !
Reference of work:
https://github.com/conda/conda/issues/9791

If you can build this recipe, then it's fine. No need to run any new tests.