[HELP ME!] Get Validator types correctly

Can anyone help me/us? I have a question about the Types of TypeScript inside Hono. This will be a BIG help. Do you know I'm creating the Validator middleware? This middleware brings type safety to Hono.
To simplify matters, I'll use pseudocode, but what I want to do is the same.
Look at this code. This is a successful pattern, and the value gotten by c.getValidatedData() has the proper type. Excellent.
type Handler<T> = (c: Context<T>) => void

class App {
  post: <T>(path: string, ...handler: Handler<T>[]) => void
}

const app = new App()

class Context<T> {
  getValidatedData: () => T
}

const validator = <T>(validatorFunc: () => T): Handler<T> => {
  return () => {}
}

app.post(
  '/',
  validator(() => {
    // return validated result values
    return {
      title: 'Hello!',
    }
  }),
  (c) => {
    const data = c.getValidatedData() // get validated result
    const { title } = data // title is `string`
    type verify = Expect<Equal<typeof title, string>>
  }
)

However, if adding a handler before the validator function, we will not get the type right.
app.post(
  '/',
  (c) => {}, // <--- Added a handler
  validator(() => {
    return {
      title: 'Hello!',
    }
  }),
  (c) => {
    const data = c.getValidatedData()
    const { title } = data <--- Error!
    type verify = Expect<Equal<typeof title, string>>
  }
)

The same problem will be held on the following test inside of the Hono validator. If we add the handler before validator, the test will fail.
https://github.com/honojs/hono/blob/465d346e4a530c11839dadc38ab558bcba275718/src/middleware/validator/index.test.ts#L23-L34
Actually, I know it's bad that the T value is fixed like the following line.
post: <T>(path: string, ...handler: Handler<T>[]) => void // <-- T is always T

This is such a difficult problem for me. Does anyone know how to solve this? I cannot sleep at night because of this problem.
Full code: https://gist.github.com/yusukebe/04445587ea5107e771a10da2c71beb53

I'm not sure but how about this?
- app.post(
+ app.post<{title: string}>(
    '/',
    (c) => {}, // <--- Added a handler
    validator(() => {
      return {
        title: 'Hello!',
      }
    }),
    (c) => {
      const data = c.getValidatedData()
      const { title } = data <--- Error!
      type verify = Expect<Equal<typeof title, string>>
    }
  )

It seems you got this error:
Argument of type 'Handler<{ title: string; }>' is not assignable to parameter of type 'Handler<unknown>'.
      Type 'unknown' is not assignable to type '{ title: string; }'.ts(2345)

That means T is treated as unknown not { title: string; } .

Hi @hogashi !
Yes, I know it, but I want to make it flexible. It has the possibility that the return values from the validator will be changed such like this:
app.post<{ title: string }>(
  '/',
  (c) => {},
  validator(() => {
    return {
      id: 123,
      title: 'Hello!',
    }
  }),
  (c) => {
    const data = c.getValidatedData()
    const { title, id } = data
    type verify = Expect<Equal<typeof title, string>>
  }
)

But, I don't want users to write app.post<{ title: string, id: number }>.
We can't predict the value returned from validator. So I use T, then it is the expected when the validator is first, but it is unkown when the handler is first.
validator is first:
app.post(
  '/',
  validator(() => {
    return {
      id: 123,
      title: 'Hello!',
    }
  }),
  (c) => {
    const data = c.getValidatedData()
    const { title, id } = data
    type verify = Expect<Equal<typeof title, string>>
    type verify2 = Expect<Equal<typeof id, number>>
  }
)

handler is first:
app.post(
  '/',
  (c) => {},
  validator(() => {
    return {
      id: 123,
      title: 'Hello!',
    }
  }),
  (c) => {
    const data = c.getValidatedData()
    const { title, id } = data
    type verify = Expect<Equal<typeof title, string>>
    type verify2 = Expect<Equal<typeof id, number>>
  }
)

This means that T becomes unknown in this line:
class App {
  post: <T>(path: string, ...handler: Handler<T>[]) => void // <--- T is `unknown`
}

Thank you for your explanation.
BTW, I tried to write down in TS Playground and it may be a different problem: https://www.typescriptlang.org/play?#code/C4TwDgpgBAEghgOwCYBsICcA8AVAfFAXigAoBjALigGEB7BYCAD2B1wEpD8A3GgSyQBQA0ijgBnMVACCYMFADeAqFDA0xLPMTBxgAC0rr0vBAHMANFAB013YlQZK8ZGix4A2gF02lHvwVLlKHQIYABXdAQAbgCAXwE44Tp1KDhZQigECAB3aVliNiERcUlaeiYNfEVAqFIk4HRQ0mAadHz-auq9XjFLExCANTgUfh0IJAARHTh0toJKoJDwhCgAchWUyWxIqBjo6rjlPuBB4aRRianKWfwt+MK6qC4hkeb0SlZiJ9OdFoAxUIQFBIHDmUGwIPwTnsrnwRGu7QWYQiwM4CjiCVSYEsqnUxACKwA9CszAEyBC0STlF8Xi1iPCqoECQTEUtHs8zgwkAsxKEUMA2ShQhAxAFlMEkcsGZ1eMA0JQVjAICgUDQAITE0U7WJsSkkUjkqU1B4c6ZEUi9Abs86TYBwNpMqBHAUvMbc3nATW1BDJeRQYAytA7dImqAO-2y6DdKAAA0MxhM0c1Do6AD0APya0CQR4YXgAMxA6QAooxIE1MEWAI6hIaYLMQGh5v0BiAWOOmXC4WICApAA
It says that title of const { title } = data becomes any . I don't know detailed behaviour but I hope this will help you.

It says that title of const { title } = data becomes any . I don't know detailed behaviour but I hope this will help you.

Thank you too. So, data become unknown and title is any. But we really want make data { title: string }!!

Hi @mizdra !

Wouldn't it solve the problem if we could distinguish between the following three types of handlers?

That's a good approach! It seems you are right about this with three types of handlers. But how about this case?

This may be mean, but it is a possibility.
We can't write the following, right?
post<T>(
  path: string,
  ...handlers1: Handler<unknown>[],
  validationHandler: ValidationHandler<T>,
  handler2: Handler<T>
): void

Wouldn't it solve the problem if we could distinguish between the following three types of handlers?

That's a good approach! It seems you are right about this with three types of handlers.

If you accept separating handlers in three, can it be object like this? (playground) Note that the handlersBeforeValidation is optional.

@hogashi
Yes, you are right, we can do it with "object". But, we don't want to use it, we don't want to break the current syntax. "path, handler1, validator, handler2" must be array.
app.post(
  '/', // <--- path
  (c) => {}, // <--- hanler1
  validator(() => { // <-- validator
    return {
      title: 'Hello!',
    }
  }),
  (c) => {}, // <--- hanler2
)

Hi @usualoma
Do you know any solution?

@yusukebe
I don't fully understand how it works, but maybe if I add Function & I don't get the error anymore.
Will this solve the problem?
https://gist.github.com/usualoma/579cbc8676b88f223c2cb3b535089f95/revisions
https://www.typescriptlang.org/play?#code/KYDwDg9gTgLgBDAnmYcCi5gGMYB4AqcoMwAdgCYDOCUArsAHxwC8c+AUKJLAsqmgEdaAQwA2uABoAaOAE0mzdnDgAKAgxUBKFk0LEyVOBLgB+OAEY4ALjgAmbfoqUlq9Vp1siIEk7mmL1nbaZjB0qDYAZmKUwOzsSChwABLCFKLAUOosqlg2AMIQpCTe6trMTABuEACW5HFYosKU1ACCYGBwAN4ukJQwAIQ2bmDCMAAWNn1Q1aQA5lIAdEtjqeTpUDYpaRnqANoAumWVNXUAvvWFfXDC7dmkwADucG1gWvWNzXAFRcRZ3cqzYAwABqYlqo2A5AAIqNhINVEc2OxzuwsJd4BUweRRtBsgQpBpMaJwTBoAAxWikXIIjz4TQ2ClUmDVQpwABkyVW6yy5S6LigQNoUFINN5nXOKJuYAWvRgKhcAHIAPQKqQuIkk6AqdxilzKAUwIUi-7KU3MmDpGwKpLAUSiCD9VV6uDnZSnTRq5QqLCIk1KpVwAB6JmdaNIV2xMGE2SwC0BIKxEOhsLeprgYaunQQ1QtqFO2UjwmdCVQFQy1QiiGyGBQOFwghE4hLEAi2dzMimM1mDAYLnOmjiUplED68uUyqdXp9HnFnrgGsjWp1TBN+sFwr5aeU5stcGttvtjrnbr7Hpc3t9Ln9QZDaYz8ELMbjQNBxMjkJhUdTpvvXTb6RdAtYWLPh53LStq0wOsGzEXBm1bHdgA7UIux7Pt2AHdgh1lMc9xVOcLxnU4COnMViPVRNSSgbVLzTA0jU3LdEKtG07QdSdTVdF0zynWi4GvYNQ3ROBH1YWN41fEkPxTAc72ErNEMA1hCxAxIy2mCDWBrbA8Bgps+Bbf8kLgTs5jQk9NCAA

Another solution would be to add another overload that supports "the two handlers before validation" pattern (playground)

Pros: Can understand why it works. I don't understand why @usualoma's solution works either.
Cons: Cannot support arbitrary number of handlers before validation. You can support 100 handlers if you write 100 overloads, but it cannot compile 101 handlers before validation. Supporting around 10 handlers might be practically acceptable though.

Note that this approach was also used in TypeScript standard library to define Promise.all before version 4.4 (see this Stack Overflow question).

In addition to that, the & Functions approach infers the handlers before validation to be (c: Context<{title: string}>) => void. This allows users to perform some operation on the context that are not validated yet, which is unsafe.

How about making the validator to accept both the validatorFunc and handler as arguments and remove generic from post method of App class:
function createValidatedHandler<T>(validator: () => T, handler: Handler<T>): Handler<T> {
  return handler
}

class App {
  post!: (path: string, ...handlers: Handler<any>[]) => void
}

Here is the playground.

How about making the validator accepts both the validatorFunc and handler as arguments and remove generic from post method of App class:
function createValidatedHandler<T>(validator: () => T, handler: Handler<T>): Handler<T> {
  return handler
}

class App {
  post!: (path: string, ...handlers: Handler<any>[]) => void
}

It's my understanding that this is a breaking change which should preferably be avoided.
One solution I can think of is to create interface with multiple tuple length definitions.
Each tuple forwards the define type from validation function
ex:
// ForwardHandlerType
type FHT<Prev> = Prev extends Handler<infer T1> ? T1 extends unknown ? unknown extends T1 ? Handler<any> : Prev : Prev : never;
interface App {
  post<A extends Handler<unknown>>(path: string, ...handler: [A]): void
  post<A extends Handler<unknown>, B extends FHT<A>>(path: string, ...handler: [A, B]): void
  post<A extends Handler<unknown>, B extends FHT<A>, C extends FHT<B>>(path: string, ...handler: [A, B, C]): void
  post<A extends Handler<unknown>, B extends FHT<A>, C extends FHT<B>, D extends FHT<C>>(path: string, ...handler: [A, B, C, D]): void
  /// ... more copy-paste...
}
class App {}
const app = new App() as App

I think there should be some way to re-write this using variadic tuple types instead https://www.typescriptlang.org/docs/handbook/release-notes/typescript-4-0.html . The key is once a Handler<T> where T != unknown is found, that T should be enforced for the rest of the Handlers

Hmmm...for this problem, it seems to me that a better approach would be to add an overload with an additional Handler<unknown>.
As in playground (https://github.com/honojs/hono/issues/823#issuecomment-1397793009), which is the approach of @mizdra and @yudai-nkt.

If there can only be one handler after validation handler, #823 (comment) can also be written like this:
type ValidationHandler<T> = Handler<T>

class App {
  post<T>(path: string, ...handlers: [ValidationHandler<T>, Handler<T>]): void
  post<T, HU extends Handler<unknown>>(path: string, ...handlers: [HU, ValidationHandler<T>, Handler<T>]): void
  post<T, HU extends Handler<unknown>>(path: string, ...handlers: [HU, HU, ValidationHandler<T>, Handler<T>]): void
  post<T, HU extends Handler<unknown>>(path: string, ...handlers: [HU, HU, HU, ValidationHandler<T>, Handler<T>]): void
  post(path: string, ...handlers: [...Handler<unknown>[]]): void {}
}

function validator<T>(validatorFunc: (c: Context<unknown>) => T): ValidationHandler<T> {
  return (c: Context<T>) => { }
}

[playground](https://www.typescriptlang.org/play?#code/KYDwDg9gTgLgBDAnmYcCi5gGMYB4AqcoMwAdgCYDOCUArsAHxwC8c+AUKJLAsqmgEdaAQwA2uABoAaOAE0mzdnDgAKAgxUBKFk0LEyVOBLgB+OAEY4ALjgAmbfoqUlq9Vp1siIEk7mmL1nbaZjB0qDYAZmKUwOzsAJakJFBRWKgAwhBJxOpwAN4uAObAMABqYvHkwiTkACLVwjbuzLrsAL5xicmpqAASwhSiwFC5BcoqWDaZ2d65mjYAbhCV7XFIKHDlopXV8Vn9g8O5rAfkQyP4DHFYosKU1ACCYGD5LpCUeJcqYNUAFjYfKCJQoyAB04N+AzOw0oNgA2lsdjA9qRTud1DI0UdLgBdeZwJYrZTvT6YgCqXh8hixI1opAA1qQIAB3UgMDQ-GD-OCA4FgiFQ86wuBw3pkmSIqrI-aC7EMTGyi4MPGLZbkN4QD4EcmUgzUGm4OmMllsjl-AGhPlwcGgyGHKDC0XiuBiiUVKUog2XBX29QqglqjVa-A6xzUxWGhlM1ns77mnmW0gg60C+2O10u50ZyW7GW+70uiO4-GE9XEzUwONci1ApP822Kx02g1G6NsuE4-2l-IdDrsCJ0nAognu6rQNwLUcwaAAMUHTUmcGmJFmrZNDG0LTY+Jz0tRRaYYzgUBKtCgpFUi+XOUum8PcF71yyHzgwmeLDgpGAzLgTzAWjiN8wFBEkVBcAByAB6cCpBcSdtilaAVAmO9XmUZQTxgM8LyPdCEHiGAhhscDemAURRAgABCGCXGUDo6M0WCXBQjwCkgyC4AAPRMWi4CwZ94ClYQPywUFijKKdgDqBoALw-jSBfPJ8MI1A2g-ITePYvDlG43j1lQBZhniCJEA-DAUBwXBBBEcR9IgCJlKGGReSTdkXA6TRAOeECKzA5QoJo8YsFQpS2lgrTdLgqckOaQ9eMw7C0O05EVOI0jyKowL0Poh9GLiIKQvYCKeLkgS4CEkSxJKXcpPqGBhFk9D5MUxzVPUhpNI47TdLw-SCSMkyzMwSzrLEXA7IclKnITWtCjcuj2E89ggJ8j4-LgAKmIK1iH3CjjIu2rdQr2rieKihCxygZDCrwhLzyS3qCKIjb0oo6jYLwnK2jy5jgtYor9pKpqyoq1hRPEmrpPqxrlGa+AlKmtrWA0vCtO6oHlD6wygUG1hzOwPBRtsvh7Na5zEzmq4Fs0IA)

This problem still remains, but it is a simple solution.
playground
@yudai-nkt @yusukebe

Hi all! Ultra-thanks for a lot of replies!
It looks like there are three ways:

using & Function
10 overloads and 10 handlers pattern
simple pattern

I think the "10 overloads" pattern is better. And for instance, we can create the rule that only ten handlers can be registered per endpoint suchapp.get('/hello').
BUT, we have another problem. There is a possibility there are multiple validators. One handler can only validate only one type from query, queries,form, json. If the user wants to validate URL params and JSON body, have to write two validators.
app.post(
  '/foo',
  validator('query', (v) => ({
    // validate URL query params
  })),
  validator('json', (v) => ({
    // validate JSON body
  })),
  (c) => {
    return c.text('Valid!')
  }
)

So I figured out the following method thanks to your hints. This would allow multiple validation results to be merged while still avoiding errors.
class App {
  post<T>(path: string, ...handlers: [Handler<T>]): void
  post<T = {}, T2 = T>(path: string, ...handlers: [Handler<T>, Handler<T2>]): void
  post<T = {}, T2 = T, T3 = T & T2>(
    path: string,
    ...handlers: [Handler<T>, Handler<T2>, Handler<T3>]
  ): void
  post<T = {}, T2 = T, T3 = T & T2, T4 = T & T2 & T3>(
    path: string,
    ...handlers: [Handler<T>, Handler<T & T2>, Handler<T & T2 & T3>, Handler<T & T2 & T3 & T4>]
  ): void
  post<T = {}, T2 = T, T3 = T & T2, T4 = T & T2 & T3, T5 = T & T2 & T3 & T4>(
    path: string,
    ...handlers: [Handler<T>, Handler<T2>, Handler<T3>, Handler<T4>, Handler<T5>]
  ): void
  post<
    T = {},
    T2 = T,
    T3 = T & T2,
    T4 = T & T2 & T3,
    T5 = T & T2 & T3 & T4,
    T6 = T & T2 & T3 & T4 & T5
    //...
  >(
    path: string,
    ...handlers: [Handler<T>, Handler<T2>, Handler<T3>, Handler<T4>, Handler<T5>, Handler<T6>]
  ): void
  post(path: string, ...handlers: [...Handler<unknown>[]]): void {
    'foo'
  }
}

This can validate a "complex pattern".

// complex pattern
app.post(
  '/',
  (c) => {},
  validator(() => {
    return {
      id: 123,
    }
  }),
  (c) => {
    const data = c.getValidatedData()
    type verify = Expect<Equal<typeof data, { id: number }>>
  },
  (c) => {},
  validator(() => {
    return {
      title: 'Hello!',
    }
  }),
  (c) => {
    const data = c.getValidatedData() // get validated result
    const { id, title } = data // id is `number`, title is `string`,
    type verify = Expect<Equal<typeof id, number>>
    type verify2 = Expect<Equal<typeof title, string>>
  }
)

Playground
How do you think this code?

Your snippet LGTM, but out of curiosity btw what do you mean by "create the rule"? Documentation, if (handlers.length > 10) {throw new Error("Too many handlers!")};, or something else?

@yudai-nkt
Thanks!

what do you mean by "create the rule"? Documentation... or something else?

Yes, I mean "rule" is writing a document about it. Like "You can't register more than 10 handlers!".

I think this issue may be resolved thanks to you. But I leave it Open for a while.

Now I will close this Issue. Thanks.


Deduplicate Issues in output

I'm not 100% sure, but it looks like multiple instances of the same issue ID cause it to appear multiple times in the changelog row.
We have PRs, commits, and branches all include out ticket number (e.g. ABC-1234) and when using the issuePattern setting, a merged PR is showing
 Pull Request #1234 : ABC-1111 Added vital changes #ABC-1111 #ABC-1111 #ABC-1111

I'm having the same problem with the same commit (hash), and looking at the results it looks like matching the tag matches v0.0.11 to v0.0.1 as well!


Missing typechecks in initOf

Currently missing arguments are ignored and compilation fails on func level

Fixed in 0.10.1


⚠ Secret Website 1 has degraded performance

In 79a37dd, Secret Website 1 ($SECRET_WEBSITE_1) experienced degraded performance:

HTTP code: 200
Response time: 6761 ms

Resolved: Secret Website 1 performance has improved in 1f63ce2.


Prisma Documentation page 404

Bug description
I got a 404 page not found when I visit https://www.prisma.io/
How to reproduce

Expected behavior
No response
Prisma information

Environment & setup

OS:
Database:
Node.js version:

Prisma Version

The website should be back up. Thanks for reporting.


Docs release for 2.19.0

Description

Update to theme customizer
Use latest W3O packages for latest features and wallet support
Update to other module docs
Fix spelling issues

Closing in favor of https://github.com/blocknative/web3-onboard/pull/1494 with working preview


Will Horiszny and Abhay Iyer Submission

Answers for Feedback 4:
- We Parsed through the JSON input and converted it into a CSV File, so that the input was compatible with Excel for Data Visualization
- Visualized demographic data, such as most languages spoken, most populated regions, etc.
- Data are in Excel sheets agents.xlsx and claims.xlsx

Hi @ReaverRage , please make sure to push up those enhancements for converting to CSV and XLSX files by 11:59 CDT. We'd love to see those visualizations!

agents.xlsx
claims.xlsx


Cli is not responding at all

I'm having an issue with now-cli. After installation now command (or any other like now -v) is not producing any output. I'm attaching a screenshot.
Using 14.0.2 on Windows 10

TLDR; "now\download\dist\now" is missing/empty.
Additional info
When installed via yarn, the "now.cmd" file contains:
@"%~dp0\..\..\..\..\..\Yarn\Data\global\node_modules\.bin\now.cmd"   %*

The referred ".bin\now.cmd" file then uses node to run "%~dp0..\now\download\dist\now", but that file is completely empty:
C:>>msls -Al node_modules\now\download\dist
total 32M
-rw------a  1 Roy              572k Mar  2 12:40 index.js
-rw------a  1 Roy                 0 Mar  2 12:46 now
-rwx-----a  1 Roy               31M Mar  2 12:46 now.exe*

Workaround, if you use scoop, which installs "now-win.exe" from the github repo (as both now-win and now) ...
C:\>scoop install "https://raw.githubusercontent.com/rivy/scoop-bucket/master/now.json"

Using scoop ...
C:\>scoop install "https://raw.githubusercontent.com/rivy/scoop-bucket/master/now.json"

This installs "now-win.exe", from the "now-cli" github repo, as both now-win and now.

rivy, this solves the problem but looks like an interesting solution, I think the bug still exists when installing cli the regular way.
BTW now getting an error about my email which is perfectly valid, but this needs its own thread I guess.

@wunnle , yep, this is is just a personal workaround, but it appears that a working fix is in progress (see #1922). I'm not sure of the timeline for the merge, but this will work until the merge occurs.

npm install -g<EMAIL_ADDRESS>while #1922 don't comes

In 14.0.3 I see the same issue.

now is empty.
$ ll /c/Users/alberto.santini/AppData/Roaming/npm/node_modules/now/download/dist/
total 20M
-rw-r--r-- 1 alberto.santini 573K Oct 26  1985 index.js
drwxr-xr-x 1 alberto.santini    0 Mar  4 10:32 ../
-rwxr-xr-x 1 alberto.santini  32M Mar  4 10:32 now.exe*
-rw-r--r-- 1 alberto.santini    0 Mar  4 10:32 now
drwxr-xr-x 1 alberto.santini    0 Mar  4 10:32 ./

And a minor detail for /c/Users/alberto.santini/AppData/Roaming/npm/now.
The first lines are repeated.
#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")

case `uname` in
    *CYGWIN*) basedir=`cygpath -w "$basedir"`;;
esac

#!/bin/sh
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")

case `uname` in
    *CYGWIN*) basedir=`cygpath -w "$basedir"`;;
esac

if [ -x "$basedir/node" ]; then
  "$basedir/node"  "$basedir/node_modules/now/download/dist/now.exe" "$@"
  ret=$?
else
  node  "$basedir/node_modules/now/download/dist/now" "$@"
  ret=$?
fi
exit $ret

Also<EMAIL_ADDRESS>has the same issue.

@AndyBitz please, shall you give another look at the issue, because it seems I can reproduce it with the latest stable and canary?

I see the same issue with 14.1.0 under git bash.

Same issue with 14.2.3 under git bash.

Here it works with now 15.2.0 via #2322.

I just updated to 15.3.0 and am still having this problem - both in Powershell and Git Bash.


Disk usage on Windows

Issue Type

[ ] Bug
[X] Feature Request
[ ] Question

Expected behavior

I would expect to get a result on Windows, if it's able to be read from a command.

Actual behavior

An error is thrown saying that the platform isn't supported.

Steps to reproduce the problem

Run si.fsStats() on Windows.

Specifications

package version: 3.42.0
operating system: Windows 10

Problem and possible solution
There is a command on Windows to read disk usage in bytes per second. This command is typeperf -si 2 -sc 1 "\LogicalDisk(_Total)\Disk Bytes/sec". Hopefully this can help you to implement this feature.

@fabiaant  - that is a good hint. Will have a look on it ...

@sebhildebrandt and @M4Yt
Just a little more on this issue i found a way to get Disk I/O and a lot of other info using typeperf using electron-sudo.
Not sure if this is something you would implement @sebhildebrandt just thought i would share it.
const Sudoer = require('electron-sudo').default;
var sudo = new Sudoer({name: 'test'});
sudo.exec('typeperf -sc 1 "\\LogicalDisk(C:)\\Disk Bytes/sec"').then(data => {
  let string = data.toString().split(/\n\s*\n/);
  string.forEach(function (item) {
    let out = item.split('\r\n');
    console.log(out[2]);
  });
}).catch(error => alert(error));

@sebhildebrandt found a better way to get Disk I/O no need for windows elevated privileges..
const util = require('../node_modules/systeminformation/lib/util');
util.powerShell('Get-Counter -Counter "\\PhysicalDisk(2 c:)\\Disk Write Bytes/sec" | Foreach-Object {$_.CounterSamples.CookedValue}').then(data => {
  console.log(data);
}).catch(error => alert(error));

Calls available:
\LogicalDisk(*)\% Free Space
\LogicalDisk(*)\Free Megabytes
\LogicalDisk(*)\Current Disk Queue Length
\LogicalDisk(*)\% Disk Time
\LogicalDisk(*)\Avg. Disk Queue Length
\LogicalDisk(*)\% Disk Read Time
\LogicalDisk(*)\Avg. Disk Read Queue Length
\LogicalDisk(*)\% Disk Write Time
\LogicalDisk(*)\Avg. Disk Write Queue Length
\LogicalDisk(*)\Avg. Disk sec/Transfer
\LogicalDisk(*)\Avg. Disk sec/Read
\LogicalDisk(*)\Avg. Disk sec/Write
\LogicalDisk(*)\Disk Transfers/sec
\LogicalDisk(*)\Disk Reads/sec
\LogicalDisk(*)\Disk Writes/sec
\LogicalDisk(*)\Disk Bytes/sec
\LogicalDisk(*)\Disk Read Bytes/sec
\LogicalDisk(*)\Disk Write Bytes/sec
\LogicalDisk(*)\Avg. Disk Bytes/Transfer
\LogicalDisk(*)\Avg. Disk Bytes/Read
\LogicalDisk(*)\Avg. Disk Bytes/Write
\LogicalDisk(*)\% Idle Time
\LogicalDisk(*)\Split IO/Sec
\PhysicalDisk(*)\Current Disk Queue Length
\PhysicalDisk(*)\% Disk Time
\PhysicalDisk(*)\Avg. Disk Queue Length
\PhysicalDisk(*)\% Disk Read Time
\PhysicalDisk(*)\Avg. Disk Read Queue Length
\PhysicalDisk(*)\% Disk Write Time
\PhysicalDisk(*)\Avg. Disk Write Queue Length
\PhysicalDisk(*)\Avg. Disk sec/Transfer
\PhysicalDisk(*)\Avg. Disk sec/Read
\PhysicalDisk(*)\Avg. Disk sec/Write
\PhysicalDisk(*)\Disk Transfers/sec
\PhysicalDisk(*)\Disk Reads/sec
\PhysicalDisk(*)\Disk Writes/sec
\PhysicalDisk(*)\Disk Bytes/sec
\PhysicalDisk(*)\Disk Read Bytes/sec
\PhysicalDisk(*)\Disk Write Bytes/sec
\PhysicalDisk(*)\Avg. Disk Bytes/Transfer
\PhysicalDisk(*)\Avg. Disk Bytes/Read
\PhysicalDisk(*)\Avg. Disk Bytes/Write
\PhysicalDisk(*)\% Idle Time
\PhysicalDisk(*)\Split IO/Sec

More ListSets available with PowerShell:
(Get-Counter -ListSet Processor).Paths
(Get-Counter -ListSet Memory).Paths

For all available Paths
(Get-Counter -ListSet *).Paths

@nrpatten Thank you!!! Great! Need to check it ... will work on it maybe this weekend.
Best Regards

@sebhildebrandt found a better way to get Disk I/O no need for windows elevated privileges..
const util = require('../node_modules/systeminformation/lib/util');
util.powerShell('Get-Counter -Counter "\\LogicalDisk(C:)\\Disk Write Bytes/sec" | Foreach-Object {$_.CounterSamples.CookedValue}').then(data => {
  console.log(data);
}).catch(error => alert(error));

Calls available:
\LogicalDisk(*)\% Free Space
\LogicalDisk(*)\Free Megabytes
\LogicalDisk(*)\Current Disk Queue Length
\LogicalDisk(*)\% Disk Time
\LogicalDisk(*)\Avg. Disk Queue Length
\LogicalDisk(*)\% Disk Read Time
\LogicalDisk(*)\Avg. Disk Read Queue Length
\LogicalDisk(*)\% Disk Write Time
\LogicalDisk(*)\Avg. Disk Write Queue Length
\LogicalDisk(*)\Avg. Disk sec/Transfer
\LogicalDisk(*)\Avg. Disk sec/Read
\LogicalDisk(*)\Avg. Disk sec/Write
\LogicalDisk(*)\Disk Transfers/sec
\LogicalDisk(*)\Disk Reads/sec
\LogicalDisk(*)\Disk Writes/sec
\LogicalDisk(*)\Disk Bytes/sec
\LogicalDisk(*)\Disk Read Bytes/sec
\LogicalDisk(*)\Disk Write Bytes/sec
\LogicalDisk(*)\Avg. Disk Bytes/Transfer
\LogicalDisk(*)\Avg. Disk Bytes/Read
\LogicalDisk(*)\Avg. Disk Bytes/Write
\LogicalDisk(*)\% Idle Time
\LogicalDisk(*)\Split IO/Sec
\PhysicalDisk(*)\Current Disk Queue Length
\PhysicalDisk(*)\% Disk Time
\PhysicalDisk(*)\Avg. Disk Queue Length
\PhysicalDisk(*)\% Disk Read Time
\PhysicalDisk(*)\Avg. Disk Read Queue Length
\PhysicalDisk(*)\% Disk Write Time
\PhysicalDisk(*)\Avg. Disk Write Queue Length
\PhysicalDisk(*)\Avg. Disk sec/Transfer
\PhysicalDisk(*)\Avg. Disk sec/Read
\PhysicalDisk(*)\Avg. Disk sec/Write
\PhysicalDisk(*)\Disk Transfers/sec
\PhysicalDisk(*)\Disk Reads/sec
\PhysicalDisk(*)\Disk Writes/sec
\PhysicalDisk(*)\Disk Bytes/sec
\PhysicalDisk(*)\Disk Read Bytes/sec
\PhysicalDisk(*)\Disk Write Bytes/sec
\PhysicalDisk(*)\Avg. Disk Bytes/Transfer
\PhysicalDisk(*)\Avg. Disk Bytes/Read
\PhysicalDisk(*)\Avg. Disk Bytes/Write
\PhysicalDisk(*)\% Idle Time
\PhysicalDisk(*)\Split IO/Sec

More ListSets available with PowerShell:
(Get-Counter -ListSet Processor).Paths
(Get-Counter -ListSet Memory).Paths

For all available Paths
(Get-Counter -ListSet *).Paths

Do you have any such command for linux as well. To get different paramters which are not available from library.


.NetCore  Microsoft. Extensions. Caching. Redis how to With asp.net Microsoft.Web. RedisSessionStateProvider Shared data?

.NetCore  Microsoft. Extensions. Caching. Redis how to With asp.net Microsoft.Web. RedisSessionStateProvider Shared data?
We have a single sign-on (sso) project, using the asp.net Microsoft.Web. RedisSessionStateProvider save the session, but the new project is using  (asp.netcore) to use the user's system, the session sharing how to do?

Start here https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/web-farm?view=aspnetcore-2.1#app-data


Version/2.11.6

Please you can merge #638 for fix 2.11.10 version on production.

I have a new bug in the production version.
For example, the return request type is changed to List<dynamic> instead of List<Pet>.
It only appears on the array return type.


Support other keyboard layouts

For a start, supporting US keyboard would be great.

Thanks for your comment. I did this some years ago as an experiment and it's not in my plans to work more on it.


Upstream updates

Coverage increased (+0.07%) to 99.02% when pulling 00664f49421a5c7125c7156978b90d290833e663 on upstream-updates into f1cce258636f7c13e17cef3c009b95b0ba6b2746 on master.


Update sed command for sklearn version

Description
sed doesn't replace sklearn version correctly, this PR aims to fix it.

@Mergifyio rebase

Might be a sporadic failure, anyway, it's not scope of this PR:
MacOS Python3.10_Sklearn1.2
=================================== FAILURES ===================================
_______ test_forest_regressor_oob[X0-y0-0.7-array-RandomForestRegressor] _______
...
>       assert regressor.oob_score_ >= lower_bound_r2
E       assert 0.6929809093349724 >= 0.7
E        +  where 0.6929809093349724 = RandomForestRegressor(n_estimators=50, oob_score=True, random_state=0).oob_score_

ensemble/tests/test_forest.py:587: AssertionError

Might be a sporadic failure, anyway, it's not scope of this PR:
MacOS Python3.10_Sklearn1.2
=================================== FAILURES ===================================
_______ test_forest_regressor_oob[X0-y0-0.7-array-RandomForestRegressor] _______
...
>       assert regressor.oob_score_ >= lower_bound_r2
E       assert 0.6929809093349724 >= 0.7
E        +  where 0.6929809093349724 = RandomForestRegressor(n_estimators=50, oob_score=True, random_state=0).oob_score_

ensemble/tests/test_forest.py:587: AssertionError

Please re-run job to check

/azp run CI


Fixes to work with stackage lts-3.1

fyi, these are some changes I made to make things work with stack and lts-3.1.

Thanks !


Crosshair line width

I cannot see a way to set the crosshair line width. Currently it is extremely thin and hard to see. Can you add a property to set the width of the crosshair line? Thanks.

Additionally a 'crosshairColor' property would be usefull, too. :o)

Is there any way to implement that? I would do that, but cannot find a way to do so. It looks like the crosshairs must be moved from the tooltip to the axis classes, and it also requires a separate crosshair class? Any idea? Thanks.

I think I found the solution and I will fix that so please do not worry about this.😝

Coooool! :o)

the default style of crosshair code in AAOptionsConstructor.m as follow
    .crosshairSet(AAObject(AACrosshair)
                  .widthSet(@2)
                  .colorSet(@"#00bfff")
                  .dashStyleSet(AACrosshairDashStyleTypeLongDashDot)
                  )

you can configure it by yourself.
postscript: dashStyle enumeration
typedef NSString *AACrosshairDashStyleType;

UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeSolid;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeShortDash;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeShortDot;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeShortDashDot;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeShortDashDotDot;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeDot;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeDash;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeLongDash;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeDashDot;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeLongDashDot;
UIKIT_EXTERN AACrosshairDashStyleType const AACrosshairDashStyleTypeLongDashDotDot;

Brilliant! Thanks a lot!


Critical error: negative currency not allowed on host with 1.3.1

BUG REPORT
Stack Trace or error message
goroutine 103800 [running]:
runtime/debug.Stack(0x446e57, 0x0, 0xc424c6a040)
/usr/local/go/src/runtime/debug/stack.go:24 +0xa7
runtime/debug.PrintStack()
/usr/local/go/src/runtime/debug/stack.go:16 +0x22
github.com/NebulousLabs/Sia/build.Critical(0xc424c6a0e8, 0x1, 0x1)
/root/go/src/github.com/NebulousLabs/Sia/build/critical.go:16 +0x93
github.com/NebulousLabs/Sia/types.Currency.Sub(0xc42110a000, 0xc4245be510, 0x2, 0x6, 0x0, 0xc4245be6f0, 0x2, 0x6, 0xc42394c000, 0xc4245be510, ...)
/root/go/src/github.com/NebulousLabs/Sia/types/currency.go:179 +0x145
github.com/NebulousLabs/Sia/modules/host.renewContractCollateral(0xc42110a000, 0x1e, 0x2a, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42394c0f0, 0x2, ...)
/root/go/src/github.com/NebulousLabs/Sia/modules/host/negotiaterenewcontract.go:44 +0x1bd
github.com/NebulousLabs/Sia/modules/host.(*Host).managedVerifyRenewedContract(0xc4200c6580, 0xc42110a000, 0x1e, 0x2a, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc42394c0f0, ...)
/root/go/src/github.com/NebulousLabs/Sia/modules/host/negotiaterenewcontract.go:266 +0x655
github.com/NebulousLabs/Sia/modules/host.(*Host).managedRPCRenewContract(0xc4200c6580, 0xde5480, 0xc42486a010, 0x0, 0x0)
/root/go/src/github.com/NebulousLabs/Sia/modules/host/negotiaterenewcontract.go:126 +0x42d
github.com/NebulousLabs/Sia/modules/host.(*Host).threadedHandleConn(0xc4200c6580, 0xde5480, 0xc42486a010)
/root/go/src/github.com/NebulousLabs/Sia/modules/host/network.go:286 +0x804
created by github.com/NebulousLabs/Sia/modules/host.(*Host).threadedListen
/root/go/src/github.com/NebulousLabs/Sia/modules/host/network.go:322 +0xc3
Critical error: negative currency not allowed
Please submit a bug report here: https://github.com/NebulousLabs/Sia/issues
General Info:
Connectability Status: Host appears to be working.
Host Internal Settings:
acceptingcontracts:   Yes
maxduration:          40 Weeks
maxdownloadbatchsize: 256.00 MB
maxrevisebatchsize:   256.00 MB
netaddress:           siad.molokanov.me:9982 (manually specified)
windowsize:           24 Hours
    collateral:       500 SC / TB / Month
    collateralbudget: 230 KS
    maxcollateral:    10 KS Per Contract

    mincontractprice:          0 H
    mindownloadbandwidthprice: 5 SC / TB
    minstorageprice:           170 SC / TB / Month
    minuploadbandwidthprice:   5 SC / TB

Host Financials:
Contract Count:               1263
Transaction Fee Compensation: 210.4 SC
Potential Fee Compensation:   375.2 SC
Transaction Fee Expenses:     0 H
    Storage Revenue:           45.05 SC
    Potential Storage Revenue: 74.03 SC

    Locked Collateral: 230.2 KS
    Risked Collateral: 4.195 KS
    Lost Collateral:   0 H

    Download Revenue:           440.1 uS
    Potential Download Revenue: 1.145 mS
    Upload Revenue:             47.32 mS
    Potential Upload Revenue:   18.97 mS

RPC Stats:
Error Calls:        36
Unrecognized Calls: 20
Download Calls:     6
Renew Calls:        17
Revise Calls:       17
Settings Calls:     1360
FormContract Calls: 13
Storage Folders:
Used         Capacity     % Used    Path
1.2043 TB    5.0997 TB    23.62     /root/go/bin/storage
root@siad:~/go/bin/host # tail host.log
2018/02/05 11:49:20.269058 errors.go:151: internal error: error with <IP_ADDRESS>:33488: incoming RPCRenewContract failed: verification of renewal failed: host has reached its collateral budget and cannot accept the file contract
2018/02/05 11:50:29.448373 errors.go:151: internal error: error with <IP_ADDRESS>:34048: incoming RPCRenewContract failed: verification of renewal failed: host has reached its collateral budget and cannot accept the file contract
2018/02/05 11:52:32.114944 errors.go:151: communication error: error with <IP_ADDRESS>:55758: incoming RPCReviseContract failed: revision iteration failed: rejected proposed modifications: unable to verify updated contract: expected at least<PHONE_NUMBER>894630670336 to be exchanged, but<PHONE_NUMBER>525078311277 was exchanged: rejected for high paying renter valid output
2018/02/05 11:57:25.681659 errors.go:151: internal error: error with <IP_ADDRESS>:41810: incoming RPCRenewContract failed: verification of renewal failed: host has reached its collateral budget and cannot accept the file contract
2018/02/05 12:02:56.385840 errors.go:151: internal error: error with <IP_ADDRESS>:39824: incoming RPCRenewContract failed: verification of renewal failed: host has reached its collateral budget and cannot accept the file contract
2018/02/05 12:07:10.541336 errors.go:151: communication error: error with <IP_ADDRESS>:33996: incoming RPCReviseContract failed: revision iteration failed: rejected proposed modifications: unable to verify updated contract: expected at least<PHONE_NUMBER>635378556928 to be exchanged, but<PHONE_NUMBER>702857971829 was exchanged: rejected for high paying renter valid output
2018/02/05 12:11:58.739583 errors.go:151: internal error: error with <IP_ADDRESS>:44030: incoming RPCRenewContract failed: verification of renewal failed: host has reached its collateral budget and cannot accept the file contract
2018/02/05 12:19:15.928095 errors.go:151: communication error: error with <IP_ADDRESS>:41256: incoming RPCReviseContract failed: revision iteration failed: rejected proposed modifications: unable to verify updated contract: expected at least<PHONE_NUMBER>005752500224 to be exchanged, but<PHONE_NUMBER>791747802104 was exchanged: rejected for high paying renter valid output
2018/02/05 12:41:02.419755 errors.go:151: communication error: error with <IP_ADDRESS>:54370: incoming RPCReviseContract failed: revision iteration failed: rejected proposed modifications: unable to verify updated contract: expected at least<PHONE_NUMBER>005752500224 to be exchanged, but<PHONE_NUMBER>791747802104 was exchanged: rejected for high paying renter valid output
2018/02/05 12:41:22.661471 errors.go:151: internal error: error with <IP_ADDRESS>:57712: incoming RPCRenewContract failed: verification of renewal failed: host has reached its collateral budget and cannot accept the file contract
Expected Behavior
How to reproduce it (as minimally and precisely as possible)
May be this is happening when the hoster is out of collateral budget.
Environment

Sia version: v1.3.1
OS: FreeBSD 11.1-STABLE #3 r327736

Redownloaded transaction pool. Still getting the same errors.

Related to #1321
I see a couple of things going on here:  The Critical error: negative currency not allowed is most concerning because that was supposedly fixed in 1.3.1.  Are you still seeing this issue and is it still crashing Sia?
The RPCRenewContract failed: verification of renewal failed: host has reached its collateral budget and cannot accept the file contract is just informational, the host ran out of collateral so you will form a new contract with a different host instead.  This is not a bug.
edit: I edited your issue to add some clearer formatting.


feature request: support virtual mappings for reduction instances

Is it possible to support virtually mapping regions with reduction privileges? Or at least support this for inner tasks only (which is all that I need). In my current use case, I have tasks that run on each node that operate on virtually mapped regions, and then launch a task to run on each GPU within the node that actually perform the desired operation. Without being able to virtually map the region for the task on each node, an unnecessary buffer is allocated and filled in system memory, and then the results of the reduction appear to be reduced into the instance in system memory from the framebuffers, even though the operation that consumes the result of the reduction uses instances in the framebuffer memory.
While it is possible to not have a hierarchical task launch structure like this, I find on other applications that this can lead to performance improvements, so I would like to keep this pattern for my applications with reduction privileges.

I think I would like to see some evidence that there's actually a performance penalty for not supporting virtual mappings on requirements with reduction privileges. Unlike normal instances, reduction instances are going to have to be folded back to a normal instance eventually anyway. Whether that happens directly or hierarchically through an intermediate copy shouldn't really matter that much, and in practice is likely to be significantly cheaper than the overhead of moving the meta-data back up the task tree for a virtual mapping.

an unnecessary buffer is allocated and filled in system memory

Why aren't you mapping this instance into the framebuffer also?

I think I would like to see some evidence that there's actually a performance penalty for not supporting virtual mappings on requirements with reduction privileges.

I have a profile here that has some evidence: http://sapling.stanford.edu/~rohany/johnson-cuda/johnson-cuda/. Disregarding the very slow CPU fill, there's a stage where reduction buffers from the child tasks on the GPU are sent back over the pcie to the buffer in system memory.
Perhaps this is an artifact of just the CPU fills being slow, but this unnecessary memory movement seems like a problem.

Unlike normal instances, reduction instances are going to have to be folded back to a normal instance eventually anyway.

It seems like for inner tasks this isn't necessarily true (at least in my case). Unless the partition for the interior task is mapped directly then my program only operates on the sub-partitions at the leaves.

Why aren't you mapping this instance into the framebuffer also?

Because there isn't enough memory to place it into a framebuffer region. The application has a node-level partition, then a partition within the nodes for each GPU (which gives it some locality within the GPUs). There isn't enough memory on any one GPU to hold the entire node level partition, which is why I want to virtually map the node level partition.

Disregarding the very slow CPU fill, there's a stage where reduction buffers from the child tasks on the GPU are sent back over the pcie to the buffer in system memory, even though the following task that really consumes the reduction buffers runs on the GPU + framebuffer.

This is an artifact of your mapping, not anything the runtime is doing. Map those reduction instances on the GPU.

It seems like for inner tasks this isn't necessarily true (at least in my case). Unless the partition for the interior task is mapped directly then my program only operates on the sub-partitions at the leaves.

That's not what I mean. Eventually all reduction instances have to be applied to a normal instance in order to be read. Where the read takes place in the region tree is irrelevant to the need to apply reduction instances back to normal instances.

The application has a node-level partition, then a partition within the nodes for each GPU (which gives it some locality within the GPUs). There isn't enough memory on any one GPU to hold the entire node level partition, which is why I want to virtually map the node level partition.

This feels like we are back to the case you had before where your logical regions are not precisely describing exactly the data that needs to be accessed by the sub-tasks.

Also, isn't it illegal to map a region into a memory that the target processor can't reach?

It is not illegal if you add the NO_ACCESS flag to the region requirement's flags.

That's surprising -- at least in my programs the region are pretty large (order of gigabytes), so I imagine that fills / data movement would dominate any runtime analysis.

Running reduction kernels on modern GPUs with 800 GB/s to 2 TB/s of memory bandwidth should require no more than a few a few hundred microseconds at most to churn through a few GB of reduction data, much faster than the 1ms on average of amortized runtime analysis overhead.

I'm pretty confused now. I don't see how to map the instances onto the GPU without running out of memory. So we're on the same page, my generated code looks like this:
a, b, c = partition matrices into tiles
index launch #1 for i, j, k in tiles onto each node:
  a, b, c = sub-partition the tiles into smaller tiles
  index launch #2 ii, jj in tiles onto each GPU in a node:
    do a gemm, reduce the result into A

At each task launch here, the bounds are as tight as I can make them. At the first index launch, the regions for a, b and c are too large to fit in a single GPU's framebuffer. Can you describe how I should be mapping the regions for index launch #1? I don't think that I can use NO_ACCESS, because doesn't that mean that task #2 can't read the regions? At least in the mapping vocabulary that I know so far, a virtual mapping seems like what I want, since I don't really need the first level tiling of a,b,c materialized in a memory.

I guess I'm wondering what value there is in doing the hierarchical decomposition of the problem like that? I can see value in doing the hierarchical decomposition if one level was dynamic (Legion) and any lower levels were static for mapping onto individual GPUs (and maybe a static recursion onto the warps, threads, etc). Figuring out the top-level dynamic decomposition as a function of the inputs should be the only thing that really needs to be dynamic I would expect.

At least for matrix multiply, I find that there is a ~10% improvement from hierarchically decomposing the problem like this on different algorithms that don't use reductions, as there is some locality in the larger tile that needs to be operated on. You're right that the top level is the only dynamic level, the lower levels are pretty much static, but I'm not sure how to expose that parallelism without launching tasks for each GPU. It does seem possible to do this with a flat index launch over all GPUs and then maybe doing a recursive task slicing, but I don't see a direct approach from my end to generate a mapper from my IR that would do this

Hey Mike, I wanted to follow up here and see what your thoughts are on this feature so I can plan around it!

It's something that I can put on the list but it probably is not going to get done until sometime in mid-2022 at the earliest. It is a very invasive change that messes with a bunch of different parts of the physical analysis.

I see. I'm shooting for the PLDI deadline on Nov 20, and having this would be necessary for some experiments that I want to run. Given that it's an invasive change can either

something hacky be done that I can keep on a separate branch for my project? I'd only need this 2-level case supported where the first level region is virtually mapped and the second level regions are mapped normally.
earlier you seemed to indicate that there was a way to map these instances to avoid the problems that I mentioned. I don't actually need virtual instances if I can avoid the allocation/fill of the instance + folding back of the instances into the node level reduction instance. If that's simpler I'll try to implement that!

something hacky be done that I can keep on a separate branch for my project?

There's no good way to hack this. You're welcome to try if you want and I'll answer questions if you do decide to go that route, but you'll be on your own in terms of writing code.

earlier you seemed to indicate that there was a way to map these instances to avoid the problems that I mentioned

I still think doing GEMM with a single-level index space task launch is going to be best for Legion. The recursive GEMM structure works much better in static cases where you are generating code.

There's no good way to hack this. You're welcome to try if you want and I'll answer questions if you do decide to go that route, but you'll be on your own in terms of writing code.

How long would you estimate this to take for someone who hasn't touched any legion internals? I could maybe give it a try if you gave me a checklist of things that needed to be done.

I still think doing GEMM with a single-level index space task launch is going to be best for Legion. The recursive GEMM structure works much better in static cases where you are generating code.

I'm waiting on some gasnet bugs to be fixed before I can really test if one is better than the other. I guess if my experiments are conclusive one way or the other then I can decide to implement this.

I still think doing GEMM with a single-level index space task launch is going to be best for Legion. The recursive GEMM structure works much better in static cases where you are generating code.

Even if that's the case, I'd still like to know if there is a way to do this mapping. Perhaps it's not a good idea for GEMM, but maybe there are other operations where it would be beneficial, which would be good if my compiler could handle it.

How long would you estimate this to take for someone who hasn't touched any legion internals? I could maybe give it a try if you gave me a checklist of things that needed to be done.

There have been two other people (one PhD student and one postdoc) who've learned enough of the Legion runtime internals to add a feature like this, and it took each of them more than a year of study and work to get to that point. If it's the only thing you worked on, I think you could probably learn enough in 6-8 months to do it, but it's not something that comes quickly in general.

Even if that's the case, I'd still like to know if there is a way to do this mapping.

Which mapping are you referring to here?

Which mapping are you referring to here?

Sorry, in response to this comment of mine above:
"earlier you seemed to indicate that there was a way to map these instances to avoid the problems that I mentioned. I don't actually need virtual instances if I can avoid the allocation/fill of the instance + folding back of the instances into the node level reduction instance. If that's simpler I'll try to implement that! The fills i'm trying to avoid are visible here -- http://sapling.stanford.edu/~rohany/johnson-bgwork/."


⚠ Fox Sports has degraded performance

In 8b7487a, Fox Sports (https://foxsports.com) experienced degraded performance:

HTTP code: 200
Response time: 711 ms

Resolved: Fox Sports performance has improved in 8fdcf6c.


Colin McRae Rally (1)

I used to be able to run the game on W10 using only this ddraw.dll and nothing else, not even a compatibility mode was required: https://www.speedrun.com/patches/DDRAW_-_fixed_ud62s.zip (comes out it is Nazroul's DDrawCompat), but that didn't work for any of my friends.
For some time applying this same dll now causes texture artifacts in menus and weird transparency problems in game.
Using dxwrapper (without DDrawCompat mode) fixes menus, but introduces back FPS drops to 0-1 in menus, making service areas a huge pain. (Especially when we want to speedrun the game) And there are still problems with transparency.
dgVoodoo seems to be doing the best job right now as it lets me play the game fine, but also causes sudden jumps of camera and randomly crashes when advancing through stages and other.
Is there any chance to get the game running?
Pictures: imgur.com/a/VKr6mLV

Here is a working build in-progress (there are still some graphical issues I am working on):  dxwrapper.zip
A couple notes:

The game is very picky about the CD drive letter.  If it is wrong it will not boot.  I recommend installing it using the CD drive letter D: and then making sure that the drive letter is set to D: any time you want to play the game.  If you have a no-CD patch this may not be needed.
I recommend setting both game.exe and rally.exe to run as admin.


Page never gets loaded when I use the plugin

Hi,
Thank you for plugin!
I have imported the plugin and using cy.preserveCookieOnce('cookiename'), but whenever I use this plugin, most of the time my page never gets load and I can see many files with 401 Unauthorized.
But Instead of the plugin when I use the below code, every thing works fine.
Cypress.Commands.add('preserveAllCookiesOnce', () => {
Cypress.Cookies.preserveOnce('cookiename');
})
I also tried debugging but that command is also not working.

Can you provide the site that is not working with the test? You could compare the cookies a normal visit sets vs preserved cookies. Maybe there are several cookies
Sent from my iPhone

On Oct 12, 2022, at 02:43, kawaljeetb @.***> wrote:

Hi,
Thank you for plugin!
I have imported the plugin and using cy.preserveCookieOnce('cookiename'), but whenever I use this plugin, most of the time my page never gets load and I can see many files with 401 Unauthorized.
But Instead of the plugin when I use the below code, every thing works fine.
Cypress.Commands.add('preserveAllCookiesOnce', () => {
Cypress.Cookies.preserveOnce('cookiename');
})
I also tried debugging but that command is also not working.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you are subscribed to this thread.

So I have further debugged the code and break my testcases into multiple testcases, and I see successful calls for api and js files now. But this is what I have understood, that when I use the plugin it doesn't preserve cookies because, when I run my 2nd testcase it gets logged out.
Below is SS with and without plugin:
with plugin:
logged in:

2nd Testcase

without plugin:

Hi. I have the same problem.
For, example, I have two tests. BeforeEach I use your plugin. In the first test, in the end, click logout, and in the second test we see the profile account but if we click some button -  we get 401 error.
And I use Cypress.Cookies.preserveOnce('cookiename') - work is correct. I click logout in the first test and second test I see a logout screen.
I have 100 passed tests with Cypress.Cookies.preserveOnce('cookiename') and if I change set cookie to your plugin 20 - 30 tests is failed


The viewer wastes too much vertical space

In general, I find that:

The top bar is way too tall, with an information density dangerously approaching zero.
The tab bars are also way too tall.
The collapsed time panel is, once again, also way too tall.
And of course the collapsed time panel should never turn into two things like it does in evidence 1 (first time I see that one to be fair).

As someone with a long-love for Vim in part because of how much it respects the user's screen estate, this is a really painful one...
Of course all of these get worse on small screens, i.e. iframes and mobiles.
My proposal would be to A) make all these bars much thinner always and maybe B) make them responsive based on the current height of the app, with some fixed minimum and maximum bounds.
(Or even, see my comment below about not having them at all :grimacing:)
Evidence 1:

Evidence 2:

Evidence 3:

Extra datapoint, a 3x3 grid in vim:

Vim famously strips away all tab bars within the grid itself to save on screen space and information density, with the assumption that you already know what is what just by the looking at the contents anyway.
In doubt, more info about the currently selected cell can be found in the one single status bar at the bottom.

Title bar
Specifically on mac, the title bar takes exactly the right vertical space in windowed mode, which is the same as any other (standard) window (though they tend to becoming increasingly rare). As soon as it goes full screen, then it's back to using too much space as per your screenshot.

Tab bar
For the tab bar, I agree, but it's not very many pixels. They can't be thiner than e.g. the ListItem. Exhibit:

With the advent of blueprint, tree drag and drop, and use on the web, I think we should also consider a mode where they are hidden altogether.
Time panel
Agree we can shave some pixel. Agree the responsive mobile mode is not ideal (unsure how to fix). A mode where it's hidden altogether is needed as well.

With the advent of blueprint, tree drag and drop, and use on the web, I think we should also consider a mode where they are hidden altogether.

Yeah I'm more and more convinced by this.

Vim famously strips away all tab bars within the grid itself to save on screen space and information density, with the assumption that you already know what is what just by the looking at the contents anyway.

I disagree on this one. This may be true for some power users/layouts. But I think having the ability to show clear titles for the space views (and containers!) is crucial for new users to understand what's going on and build the link between the blueprint tree and the layout itself.
Also the assertion that you know what's there by the contents only holds when things are working well and are visually distinct. Users spend lots and lots of time with space views not showing what they're supposed to. Even if configured properly, a key use case of rerun is helping users understand when their own code is not producing the anticipated results.
Furthermore, Vim is famously unfriendly to new users -- I don't think this is the right choice for us to emulate.

A good example why this matters is the https://www.yaak.ai/Research web site:


Corrections from feedback2.0

Corrections from feedback of 2.0.0

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

285 of 314 (90.76%)  changed or added relevant lines in 14 files are covered.
28 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-11.07%) to 88.932%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

repository/IngSoft2-Model/HyperJumpEffect.class.st
29
34
85.29%

repository/IngSoft2-Model/Board.class.st
55
79
69.62%

Files with Coverage Reduction
New Missed Lines
%

repository/IngSoft2-Model/Board.class.st
28
60.31%

Totals

Change from base Build<PHONE_NUMBER>:
-11.07%

Covered Lines:
458

Relevant Lines:
515

💛  - Coveralls


Fix version updating

proper fix for #3039

does the link generated to the release still work?

Thats why i'm for  #3042
It also fixes other edgecases too


ocamlPackages.checkseum: 0.0.3 → 0.1.1

Motivation for this change
This fixes the “decompress” library on non-x86_64 platforms
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build ocamlPackages.decompress


Module installation with Drupal 11

Currently, the Drupal version to install is determined in expand-composer-json based on $DDEV_PROJECT_TYPE.
It looks like the current approach won't work for Drupal 11 when DDEV introduces D11 support, as $DDEV_PROJECT_TYPE won't have a drupal11 value, but will simply be drupal.

DDEV does detect and use the Drupal version though. Should it be introduced as a new variable or something?

If a contrib module supports (or aims to support) both D10 and D11, what would be the installation flow to explicitly install Drupal 11 instead of a Drupal 10?
To achieve this now, I have manually added [[ $DDEV_PROJECT_TYPE == "drupal" ]] && export _TARGET_CORE=11.x-dev to the expand-composer-json file - I am just not sure if there is a better way than this "hack".

There is another approach that does not involve modifying composer.json:
For example, in .ddev/commands/web/poser', just before composer install', we can add
composer require drupal/core-recommended:11.x-dev@dev drupal/core-composer-scaffold:11.x-dev@dev drupal/core-project-message:11.x-dev@dev --update-with-all-dependencies to install Drupal 11.x-dev.
As a side note, I was getting the error 'error This project's package.json defines "packageManager"<EMAIL_ADDRESS>However, the current global version of Yarn is 1.22.22'. - To fix this, I also added yarn policies set-version stable to the poser file.

@mscieszka Thanks for you quick work!  I applied your fix to .ddev/commands/web/poser, but I had to add the latest version of drush to the composer require line.
Here's the edited poser file.
.ddev/commands/web/expand-composer-json "$DDEV_PROJECT_NAME"
composer require drupal/core-recommended:11.x-dev@dev drupal/core-composer-scaffold:11.x-dev@dev drupal/core-project-message:11.x-dev@dev drush/drush:13.x-dev --update-with-all-dependencies
composer install
rm composer.contrib.json composer.contrib.lock
yarn policies set-version stable
yarn --cwd $DDEV_DOCROOT/core install
touch $DDEV_DOCROOT/core/.env```

I took a different direction than #45. See my PR in #46. Feedback welcome.

Fixed in #46


[FLINK-6130] [yarn] Fix Consider calling resourceManager#getTerminati…

…onFuture() with lock held.
Thanks for contributing to Apache Flink. Before you open your pull request, please take the following check list into consideration.
If your changes take all of the items into account, feel free to open your pull request. For more information and/or questions please refer to the How To Contribute guide.
In addition to going through the list, please provide a meaningful description of your changes.

[ ] General

The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
The pull request addresses only one issue
Each commit in the PR has a meaningful commit message (including the JIRA id)

[ ] Documentation

Documentation has been added for new functionality
Old documentation affected by the pull request has been updated
JavaDoc for public methods has been added

[ ] Tests & Build

Functionality added by the pull request is covered by tests
mvn clean verify has been executed successfully locally or a Travis build has passed

@tedyu @zentol It seems there is a problem with travis-ci test if no output within 5 mins then it will print the stacktrace and then kill the watchdog , please help to review and checkout how to solve, I am appreciate it.
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.213 sec - in org.apache.flink.runtime.minicluster.MiniClusterITCase
==============================================================================
Maven produced no output for 300 seconds.
==============================================================================
==============================================================================
The following Java processes are running (JPS)
==============================================================================
1552 Launcher
12642 Jps
8045 surefirebooter5889190340308801783.jar
...
Trying to KILL watchdog (1547).
MVN exited with EXIT CODE: 143.
java.io.FileNotFoundException: build-target/lib/flink-dist*.jar (No such file or directory)
	at java.util.zip.ZipFile.open(Native Method)
	at java.util.zip.ZipFile.<init>(ZipFile.java:220)
	at java.util.zip.ZipFile.<init>(ZipFile.java:150)
	at java.util.zip.ZipFile.<init>(ZipFile.java:121)
	at sun.tools.jar.Main.list(Main.java:1060)
	at sun.tools.jar.Main.run(Main.java:291)
	at sun.tools.jar.Main.main(Main.java:1233)
find: `./flink-yarn-tests/target/flink-yarn-tests*': No such file or directory
PRODUCED build artifacts.

@tzulitai Hi, Could you please review my code ? I am appreciate it.

I don't think the timeout was related.

What is the problem we're trying here to solve?

@StephanEwen Hi, How do you think of this refine ?

@zentol Please helps review. I am very appreciate it.

@zhangminglei please refrain from pulling more and more people into the loop by directly referencing them given that this is a trivial change and that 5 people are already involved.
Looking at the YarnFlinkApplicationMasterRunner I think it could benefit from some more refactoring. For example, I don't think that we need to have an AbstractYarnFlinkApplicationMasterRunner base class. Both can be combined in one class. Then we can instantiate the services and runtime components in the constructor of YarnFlinkApplicationMasterRunner. That way we can get rid of the lock in the run method.
Given that, I'm not sure whether this PR makes then sense anymore. I think we can close it.

@tillrohrmann That is right. We can close this PR surely. BTW, Should we make another jira refactoring this class and AbstractYarnFlinkApplicationMasterRunner  ? Because I watch the AbstractYarnFlinkApplicationMasterRunner  and find only one subclass. If we do create another jira, You are the reporter and assign it to me. How do you think of this ?

Yes we should create another JIRA issue. You can create it. You can also close FLINK-6130 then.


Upward-looking util function

We could make some upward-looking util function that can be used to look for a certain file or directory in the current directory and search parent directories if that file or folder is not found. Currently, the tool would not work when run from a subfolder.
This is especially true for the module lookup functionality.

Originally posted by @mitchellolsthoorn in https://github.com/syntest-framework/syntest-core/pull/234#discussion_r1117141969

We can use the package https://www.npmjs.com/package/find-up to achieve the desired functionality.

Similarly, we can use the same package to improve the config file behavior by applying upward looking there as well.
In the option, you would need to add:
configParser: function (configPath) {
    return JSON.parse(fs.readFileSync(configPath, 'utf-8'))
  }

This way we can support multiple config formats and allow users to choose both a specific config and an upward look if no option is provided


CircleCI doesn't Trigger on Updating Submodule

Circleci doesn't run the compilation and tests if we only change the submodule (e.g. point to the latest commit in the submodule). We want circleci to be triggered even if anything doesn't change in aztec3-circuits but any submodule is changed.

Just notices that if I push a commit that updates a submodule, circleci wrongly shows a green tick ✅ when it should compile everything from scratch. (It just refers to some old green tick and avoids running any tests)

@suyash67 is this still an issue?

Closing this as I believe I know what was causing this a while back (issue with the way our CI scripts were looking for submodule bumps) and it has been fixed.


update of jquery and bootstrap shipped with mokey

Hi,
after a security scan on our instance of mokey we have found out that the jquery and bootstrap versions shipped with mokey are outdated. Is it possible to include the newest version in more recent releases?
Do you think it is possible to just replace the existing version with the newest one? Of course we are going to try it but since you have bigger insight of the project.
I had a quick look and bootstrap seems to be easier while upgrading jquery might be more challenging as migration from current version (1.11) to latest version (3.6) requires a couple of intermediate steps...
Best Regards,
Marcello

@mmorgotti Thanks for reporting this. Would be great to get bootstrap/jquery updated. If you end up getting it working feel free to submit a PR. We'll try and bump this up on the priority list.

As our Security Scanners reported the same I looked into things.
About jQuery; You can see it as a „drop-in replacement“ and everything remains working it seems. That‘s at least what I‘ve done and we’ve not seen and issues so far.
Bootstrap is a little more efforts as a significant number of CSS classes were updated, but I started to look into that. May be I‘ll find some time during holidays.

@hexa2k9 Sounds great! Let us know if you get PR going. This has been on the list for a while now and would be great to get updated.

fixed in v0.6.0


Create Technical Assessment for BASICS Testing and Release Manager - 2021-06-08

Steps to resolve:
If necessary, read the retrospective:
https://cryptpad.fr/pad/#/2/pad/edit/lJpkhqY+4USDfbtLpD47NxPC/
Enter your thoughts into this Cryptpad:
https://cryptpad.fr/pad/#/2/pad/edit/vj1oTy+a9hGmSqyQGaVvebwV/
At the meeting on 2021-06-04, we will decide on assessment which will then be emailed to BASICS that night.

Assessment sent to BASICS; will be added to repo after candiate is chosen.

Assessing Candidate Technical Ability in 15 Minutes or Less
During the hiring of a Testing a Release Manager, in conjunction with BASICS/Internews, the Tahoe-LAFS team designed a means to assess the technical ability of the candidates.
We do not support take-home assignments, since they ask candidates for unpaid labor and can exacerbate biases in the hiring process. However, it was important to know if the candidate would be capable of working on the Tahoe-LAFS codebase, and to what degree.
Our solution was to create a strictly timed exercise, of 15 minutes only, in the form of a real-time screenshare. One member of the Development team and the Community Manager (who took notes and did introductions), sat in on the exercise.
The Process

Preparation: Candidates should have installed:

A shell
IRC client
Git client
Python 2.7

Assignment (instructions given at the appointment; candidate received a link to a list of the tasks): Candidates will screenshare for 15 minutes, and complete as many of the tasks below as possible. Tasks are not to be done in any particular order, and we DO NOT expect candidates to be able to complete all tasks. Candidates should not expect assistance from the sitter (one of our devs), but they can ask the sitter for clarification on any of the tasks. Candidates will not know about tasks beforehand.

download the Tahoe-LAFS code locally (i.e. "git clone")
determine what IRC network we use
connect to the IRC network we use
say something in the #tahoe-lafs channel in the IRC network we use
create a Python virtualenv
install the Tahoe-LAFS code previously downloaded into the virtualenv
add your name to the contributors list (locally)
propose a PR which adds you to the contributors list

The sitter might leave their camera off, but their mic on. The sitter has been informed that this situation might be stressful to candidates, so foresight has been taken to alleviate stressors. Friendly tone, helpful demeanor. We will not penalize for internet issues, and strive to make accommodations/reschedule.

Result
Only one candidate was able to complete all of the tasks within 15 minutes. If no candidates has been able to complete all of the tasks, we still would have collected valuable insight into candidates' ability to execute on these tasks. After a real-time informational interview this candidate was hired, with ultimate results TBD.


Housekeeping

Bunch of minor housekeeping

Update the workspace resolver
Add check to CI that our stated MSRV is accurate
Incidentally, had to bump the MSRV. I don't think it was accurate before, and it definitely wasn't after upgrades. If anyone has concerns about bumping all the way to 1.74, we could downgrade clap one minor version to get the MSRV down to 1.70. I checked Martin and it looks to already target 1.74, and internally at Stadia we target latest stable.
Add cargo semver checks to CI to ensure we don't accidentally break API compatibility
Upgrade clap (major change; I went for the derive API as it's a great fit for this use case)
Upgrade other dependencies, most notably freetype-rs

sorry, got swamped, will review shortly too


Move Storage to Async

Closes/fixes #609
Checklist

[X] This PR makes changes to the public API
[X] I have included links for closing relevant issue numbers

@Badisi Can you take care of reviewing this MR?

@pamapa, sure! I was actually going to leave a comment ;-)
I don't know about you but I think it is a little bit too much modifications.
As I said in the original issue, the WebStorageStateStore can already support an async store.
If we take this for example:
const item = this._store.getItem(key);
return Promise.resolve(item);

item could be a simple object/value (sync) as well as a Promise (async), and Promise.resolve will properly resolve both cases.
The only issue here is that _store is of type Storage (ie. sync). So I would just create and AsyncStorage type and simply change the store type to Storage | AsyncStorage.

Here is what I would do with this MR:

[ ] keep the InMemoryWebStorage class (do not renaming it to avoid breaking changes for people that might already use it)
[ ] remove AsyncInMemoryWebStorage and AsyncLocalStorage (the library should not be providing every type of storage that might exists, it should be the developer responsibility to create the one that fits his needs using the Storage or AsyncStorage types)
[ ] revert back OidcClientSettings, UserManagerSettings and WebStorageStateStore
[ ] change WebStorageStateStore._store type to Storage | AsyncStorage
[ ] modify WebStorageStateStore#L25 so that the result is returned by Promise.resolve

@Badisi @pamapa I've reverted a lot of changes whilst keeping the new async storage option. Let me know if these are satisfactory.
Regarding the comment around returning Promise.resolve inside an async method, this doesn't work (Typescript complains) and isn't needed?

@philjones88, looks way simpler thanks !
And is it still working like this for your use case ?
You are right it wasn't needed as the set() method returns void, my bad ^^

@Badisi I believe so, it works with my scenario.
For anyone else trying to use this in an Electron React app:
For Electron I have this in my client:
      stateStore: new StateStoreElectron(),
      userStore: new WebStorageStateStore({ prefix: 'oidc.', store: new AsyncStoreElectron() }),

Where StateStoreElectron is:
import type { StateStore } from 'oidc-client-ts';
import { getLogger } from 'logger;

class StateStoreElectron implements StateStore {
  private readonly _logger = getLogger('StateStoreElectron');

  private readonly _prefix: string;

  private readonly _instance: ElectronWindow;

  public constructor({ prefix = 'oidc.' } = {}) {
    const instance = window.electron || window.parent.electron;

    if (!instance) {
      throw new Error('Window instance is undefined');
    }

    this._prefix = prefix;
    this._instance = instance;
  }

  public async set(key: string, value: string): Promise<void> {
    this._logger.debug({ key, value }, 'set');

    const keyWithPrefix = this._prefix + key;
    await this._instance.setStoreValue(keyWithPrefix, value);
    return Promise.resolve();
  }

  public async get(key: string): Promise<string | null> {
    this._logger.debug({ key }, 'get');

    const keyWithPrefix = this._prefix + key;
    const item = await this._instance.getStoreValue(keyWithPrefix);
    return Promise.resolve(item || null);
  }

  public async remove(key: string): Promise<string | null> {
    this._logger.debug({ key }, 'remove');

    const keyWithPrefix = this._prefix + key;
    const item = await this._instance.getStoreValue(keyWithPrefix);
    await this._instance.removeStoreKey(keyWithPrefix);
    return Promise.resolve(item || null);
  }

  public async getAllKeys(): Promise<string[]> {
    this._logger.debug({}, 'getAllKeys');

    const keys = await this._instance.getStoreKeys();
    return Promise.resolve(keys || []);
  }
}

export default StateStoreElectron;

and AsyncStoreElectron is:
/* eslint-disable no-underscore-dangle */
import { AsyncStorage } from 'oidc-client-ts';

class AsyncStoreElectron implements AsyncStorage {
  private readonly _instance: ElectronWindow;

  public get length(): Promise<number> {
    if (!window.electron) {
      return Promise.resolve(0);
    }
    return this._instance.getStoreLength();
  }

  constructor() {
    const instance = window.electron || window.parent.electron;

    if (!instance) {
      throw Error('electron on window is undefined. Are you running in Electron?');
    }

    this._instance = instance;
  }

  public async clear(): Promise<void> {
    await this._instance.clearStore();
  }

  public async getItem(key: string): Promise<string | null> {
    const item = await this._instance.getStoreValue(key);
    return item || null;
  }

  public async key(index: number): Promise<string | null> {
    const item = await this._instance.getStoreKey(index);
    return item || null;
  }

  public async removeItem(key: string): Promise<void> {
    await this._instance.removeStoreKey(key);
  }

  public async setItem(key: string, value: string): Promise<void> {
    await this._instance.setStoreValue(key, value);
  }
}

export default AsyncStoreElectron;

The window.electron and window.parent.electron is because inside my preload file I have:
contextBridge.exposeInMainWorld('electron', { ... });
This allows me to use electron-store to handle persistence in Electron via IPC.
Annoyingly the way OIDC-Client-TS makes iframes causes issues, so I'm having to check both the current window and parent to try and find the electron properties.

thanks for the contribution and thanks for reviewing!

Just wanted to double check
new WebStorageStateStore({ store: window.localStorage }

localStorage is not async interface so I need to change it with new version, or?
Thanks!

Great, thanks for clarifying. Great update!


DetectNet_v2 tensorrt License Plate Detection (LPDNet)

Hi,
I have made a tensorrt engine of the model downloadable from here:
tlt-converter -k nvidia_tlt -d 3,480,640 -p image_input,1x3x480x640,4x3x480x640,16x3x480x640 usa_pruned.etlt -t fp16 -e lpd_engine.trt
The model is based off of DetectNet_v2. Has anyone managed to get this or a other DetectNet_v2 model working with tensorrt in python?
Here is the code I have so far:
import os
import time

import cv2
#import matplotlib.pyplot as plt
import numpy as np
import pycuda.autoinit
import pycuda.driver as cuda
import tensorrt as trt
import pdb

class HostDeviceMem(object):
    def __init__(self, host_mem, device_mem):
        self.host = host_mem
        self.device = device_mem

    def __str__(self):
        return "Host:\n" + str(self.host) + "\nDevice:\n" + str(self.device)

    def __repr__(self):
        return self.__str__()

def load_engine(trt_runtime, engine_path):
    with open(engine_path, "rb") as f:
        engine_data = f.read()
    engine = trt_runtime.deserialize_cuda_engine(engine_data)
    return engine

# Allocates all buffers required for an engine, i.e. host/device inputs/outputs.
def allocate_buffers(engine, batch_size=1):
    inputs = []
    outputs = []
    bindings = []
    stream = cuda.Stream()
    for binding in engine:
        # pdb.set_trace()
        size = trt.volume(engine.get_binding_shape(binding)) * batch_size
        dtype = trt.nptype(engine.get_binding_dtype(binding))
        # Allocate host and device buffers
        host_mem = cuda.pagelocked_empty(size, dtype)
        device_mem = cuda.mem_alloc(host_mem.nbytes)
        # Append the device buffer to device bindings.
        bindings.append(int(device_mem))
        # Append to the appropriate list.
        if engine.binding_is_input(binding):
            inputs.append(HostDeviceMem(host_mem, device_mem))
            print(f"input: shape:{engine.get_binding_shape(binding)} dtype:{engine.get_binding_dtype(binding)}")
        else:
            outputs.append(HostDeviceMem(host_mem, device_mem))
            print(f"output: shape:{engine.get_binding_shape(binding)} dtype:{engine.get_binding_dtype(binding)}")
    return inputs, outputs, bindings, stream

def do_inference(context, bindings, inputs, outputs, stream, batch_size=1):
    # Transfer input data to the GPU.
    [cuda.memcpy_htod_async(inp.device, inp.host, stream) for inp in inputs]
    # Run inference.
    context.execute_async(
        batch_size=batch_size, bindings=bindings, stream_handle=stream.handle
    )
    # Transfer predictions back from the GPU.
    [cuda.memcpy_dtoh_async(out.host, out.device, stream) for out in outputs]
    # Synchronize the stream
    stream.synchronize()
    # Return only the host outputs.
    return [out.host for out in outputs]

# TensorRT logger singleton
os.environ["CUDA_VISIBLE_DEVICES"] = "1"
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt_engine_path = "lpd.trt"

trt_runtime = trt.Runtime(TRT_LOGGER)
# pdb.set_trace()
trt_engine = load_engine(trt_runtime, trt_engine_path)
# Execution context is needed for inference
context = trt_engine.create_execution_context()
# This allocates memory for network inputs/outputs on both CPU and GPU
inputs, outputs, bindings, stream = allocate_buffers(trt_engine)

# pdb.set_trace()
image = cv2.imread("car.jpg")
image = cv2.resize(image, (640, 480))

np.copyto(inputs[0].host, image.ravel())

outputs = do_inference(context, bindings=bindings, inputs=inputs, outputs=outputs, stream=stream)
test1 = np.reshape(outputs[0], (4, 30, 40))
test2 = np.reshape(outputs[1], (1, 30, 40))
print(outputs)

Exporting the tensorrt engine and then running this code works. I am just unable to interpret this output.
The output is supposedly a

40x30x12 bbox coordinate tensor and 40x30x3 class confidence tensor.

So I then reshape it into these dimensions but the bbox coordinate tensor has decimals which I can't convert to pixel coords.
I hope this is relevant to the repo - I think it could be a nice addition once it works.

Exporting the tensorrt engine and then running this code works. I am just unable to interpret this output.

I have not used DetectNet_v2 yet, but I think this should be easy to fix.  Is there any DetectNet_v2 inference code we can reference?  This code could be in either python or C++ or else.

yes, check this link https://github.com/dusty-nv/jetson-inference/blob/master/c/detectNet.cpp#L815

The LPDnet docs mention:

The raw normalized bounding-box and confidence detections need to be post-processed by a clustering algorithm such as DBSCAN or NMS to produce the final bounding-box coordinates and category labels.

I tried sklearns DBSCAN clustering but it returns an array of 0's.

You need to fix the preprocessing code first.  Refer to: https://github.com/dusty-nv/jetson-inference/blob/19ed62150b3e9499bad2ed6be1960dd38002bb7d/c/detectNet.cpp#L729
The input tensor for DetectNet is: CHW ordered, RGB, float32, ranged from -1.0 to +1.0.
So you should do something like this:  (assuming 640x480 is the correct dimension of DetectNet input)
image = cv2.imread("car.jpg")
image = cv2.resize(image, (640, 480))
image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)         # BGR -> RGB
image = image.transpose((2, 0, 1)).astype(np.float32)  # HWC -> CHW, uint8 -> float32
image = (image - 127.5) / 127.5

np.copyto(inputs[0].host, image.ravel())

As to postprocessing, refer again to DetectNet code in jetson_inference repo: https://github.com/dusty-nv/jetson-inference/blob/19ed62150b3e9499bad2ed6be1960dd38002bb7d/c/detectNet.cpp#L813-L861
Output tensor 0 should be "conf", while output tensor 1 "bbox".  So I think you should do something like:
confs = np.reshape(outputs[0], (-1, 3))
bboxes = np.reshape(outputs[1], (-1, 4))

Where "confs" are confidence scores of the 3 classes for each potential detection, and "bboxes" are coordinates of the potential detections.  More specifically, "bboxes" are "(x1, y1, x2, y2)" (or ("Left", "Top", "Right", "Bottom")) coordinates ranged from 0.0 to 1.0.  You'll need to multiply them with image width and height, e.g. 640 and 480, to get the pixel coordinates on the original image.
I hope this helps and you'll be able to fix the code by yourself.

Thanks for the help. Viewing the confidence output matches perfectly with the image.
The bbox output I don't quite understand. Is this the correct process?
Use the confidence tensor (30, 40) to select cells above a confidence threshold.
Then in the bbox tensor (4, 30, 40) use the selected cells to get the 4 corner coords.
In the LPD_net documentation they mention a "40x30x12 bbox coordinate tensor" but the array won't reshape to that as it has a length of 4800.

I should be able to fix it myself now.
Appreciate the help.

Sorry to open this again. I am struggling to deal with outputs not being normalised.
The confidence array looks right with this confs = np.reshape(outputs[1], (-1, 3)) I assume the shorter outputs[n] will be confidence. This array is between 0 and 1.
The bboxes bboxes = np.reshape(outputs[0], (-1, 4)) has values ranging from -0.1 to 5. With the following test image:

The lengths of the two output arrays also dont match up when reshaped with 3 and 4. len(outputs[0]) = 4800 and len(outputs[1]) = 1200.
confs = np.reshape(outputs[1], (-1, 3)) becomes (400, 3)
bboxes = np.reshape(outputs[0], (-1, 4)) becomes (1200, 4)
The code I'm using is below:
image = cv2.imread("img.png")
image_orig = cv2.resize(image, (640, 480))

image = cv2.cvtColor(image_orig, cv2.COLOR_BGR2RGB)  # BGR -> RGB
image = image.transpose((2, 0, 1)).astype(np.float32)  # HWC -> CHW, uint8 -> float32
image = (image - 127.5) / 127.5

np.copyto(inputs[0].host, image.ravel())

outputs = do_inference(context, bindings=bindings, inputs=inputs, outputs=outputs, stream=stream)
confs = np.reshape(outputs[1], (-1, 3))
bboxes = np.reshape(outputs[0], (-1, 4))

result = np.where(confs > 0.3)
for index in result[0]:
    rect = bboxes[index]
    test = cv2.rectangle(image_orig, (int(rect[0] * 640), int(rect[1] * 480)), (int(rect[2] * 640), int(rect[3] * 480)),
                         (0, 0, 255), 2)

cv2.imshow("1", test)

Based on your description, I realized that the number of object classes is only 1 ("license plate").  So you should reshape confs as (-1, 1) array instead.
confs = np.reshape(outputs[1], (-1, 1))   # (1200, 1)
bboxes = np.reshape(outputs[0], (-1, 4))  # (1200, 4)

In addition, since your original image is 640x430.  You should use 430 as height in this line of code.
    test = cv2.rectangle(image_orig, (int(rect[0] * 640), int(rect[1] * 430)), (int(rect[2] * 640), int(rect[3] * 430)),
                         (0, 0, 255), 2)

Oh yes that makes sense.
Do you perhaps know how to deal with the bboxes with a rangenot between 0 and 1? The boxes are now drawn off the image as some values are negative and some above 1.
is it possible this line https://github.com/dusty-nv/jetson-inference/blob/19ed62150b3e9499bad2ed6be1960dd38002bb7d/c/detectNet.cpp#L846 causes this issue? I can't see how it could make the value smaller or positive but it seems I am missing this line.

is it possible this line https://github.com/dusty-nv/jetson-inference/blob/19ed62150b3e9499bad2ed6be1960dd38002bb7d/c/detectNet.cpp#L846 causes this issue? I can't see how it could make the value smaller or positive but it seems I am missing this line.

Nope.  That line is only for calculating the pointer to the corresponding bboxes quadruple.  Your python code is matching what detectNet.cpp is doing.

Do you perhaps know how to deal with the bboxes with a range not between 0 and 1?

You only need to care about bboxes values with which the corresponding conf score is over the threshold.  Could you try to print out those values?

I should've sent the output so far. If you ignore the low confidence values the bounding boxes are all within 0 and 1 - I thought this was coincidental.

Could it be that this model is just terrible (above confidence is ~0.5)? The other images have a max confidence of 0.1:

The model specs on the Nvidia container registry seem very incorrect.
Model | Dataset | Accuracy
-- | -- | --
usa_unpruned_model | NVIDIA 3k LPD eval dataset | 98.58%
usa_pruned_model | NVIDIA 3k LPD eval dataset | 98.46%  <- the one I am using
ccpd_unpruned_model | 14% of CCPD-Base dataset | 99.24%
ccpd_pruned_model | 14% of CCPD-Base dataset | 99.22%

The full working code thus far for anyone who wanders onto this issue
import os
import cv2
import numpy as np
import pycuda.autoinit
import pycuda.driver as cuda
import tensorrt as trt

class HostDeviceMem(object):
    def __init__(self, host_mem, device_mem):
        self.host = host_mem
        self.device = device_mem

    def __str__(self):
        return "Host:\n" + str(self.host) + "\nDevice:\n" + str(self.device)

    def __repr__(self):
        return self.__str__()

def load_engine(trt_runtime, engine_path):
    with open(engine_path, "rb") as f:
        engine_data = f.read()
    engine = trt_runtime.deserialize_cuda_engine(engine_data)
    return engine

# Allocates all buffers required for an engine, i.e. host/device inputs/outputs.
def allocate_buffers(engine, batch_size=1):
    inputs = []
    outputs = []
    bindings = []
    stream = cuda.Stream()
    for binding in engine:
        # pdb.set_trace()
        size = trt.volume(engine.get_binding_shape(binding)) * batch_size
        dtype = trt.nptype(engine.get_binding_dtype(binding))
        # Allocate host and device buffers
        host_mem = cuda.pagelocked_empty(size, dtype)
        device_mem = cuda.mem_alloc(host_mem.nbytes)
        # Append the device buffer to device bindings.
        bindings.append(int(device_mem))
        # Append to the appropriate list.
        if engine.binding_is_input(binding):
            inputs.append(HostDeviceMem(host_mem, device_mem))
            print(f"input: shape:{engine.get_binding_shape(binding)} dtype:{engine.get_binding_dtype(binding)}")
        else:
            outputs.append(HostDeviceMem(host_mem, device_mem))
            print(f"output: shape:{engine.get_binding_shape(binding)} dtype:{engine.get_binding_dtype(binding)}")
    return inputs, outputs, bindings, stream

def do_inference(context, bindings, inputs, outputs, stream, batch_size=1):
    # Transfer input data to the GPU.
    [cuda.memcpy_htod_async(inp.device, inp.host, stream) for inp in inputs]
    # Run inference.
    context.execute_async(
        batch_size=batch_size, bindings=bindings, stream_handle=stream.handle
    )
    # Transfer predictions back from the GPU.
    [cuda.memcpy_dtoh_async(out.host, out.device, stream) for out in outputs]
    # Synchronize the stream
    stream.synchronize()
    # Return only the host outputs.
    return [out.host for out in outputs]

# TensorRT logger singleton
os.environ["CUDA_VISIBLE_DEVICES"] = "1"
TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt_engine_path = "lpd.trt"

trt_runtime = trt.Runtime(TRT_LOGGER)
# pdb.set_trace()
trt_engine = load_engine(trt_runtime, trt_engine_path)
# Execution context is needed for inference
context = trt_engine.create_execution_context()
# This allocates memory for network inputs/outputs on both CPU and GPU
inputs, outputs, bindings, stream = allocate_buffers(trt_engine)

# pdb.set_trace()
image = cv2.imread("img_2.png")
image_orig = cv2.resize(image, (640, 480))

image = cv2.cvtColor(image_orig, cv2.COLOR_BGR2RGB)  # BGR -> RGB
image = image.transpose((2, 0, 1)).astype(np.float32)  # HWC -> CHW, uint8 -> float32
image = (image - 127.5) / 127.5

np.copyto(inputs[0].host, image.ravel())

outputs = do_inference(context, bindings=bindings, inputs=inputs, outputs=outputs, stream=stream)
confs = np.reshape(outputs[1], (-1, 1))
bboxes = np.reshape(outputs[0], (-1, 4))

scores = []

result = np.where(confs > 0.5)
for index in result[0]:
    rect = bboxes[index]
    test = cv2.rectangle(image_orig, (int(rect[0] * 640), int(rect[1] * 430)), (int(rect[2] * 640), int(rect[3] * 430)),
                         (0, 0, 255), 2)

cv2.imshow("1", test)

cv2.waitKey()
print(outputs)

confs = np.reshape(outputs[1], (-1, 1))
bboxes = np.reshape(outputs[0], (-1, 4))

Something is not right here...
OUTPUT_CONF is index 0 and OUTPUT_BBOX is 1 in jetson_inference code.  And the postprocessing code reads:
		float* conf = mOutputs[OUTPUT_CONF].CPU;
		float* bbox = mOutputs[OUTPUT_BBOX].CPU;

The output indices do not match with your code.  Are you sure the LPDNet model you're using could be processed by jetson_inference code??

Yeah, I also noticed that but the dimensions of the outputs only make sense the way it is now.
If the model is for tensorrt does that mean it will work with jetson_inference? From the models documentation:

These models need to be used with NVIDIA Hardware and Software. For Hardware, the models can run on any NVIDIA GPU including NVIDIA Jetson devices. These models can only be used with Transfer Learning Toolkit (TLT), DeepStream SDK or TensorRT.

But thank you for your time. I've managed to train a yolov4 license plate detector in the mean time that works sufficiently and will work with this library.

@C-monC I am facing similar issues with DetectNet_v2. According to your last comment were you using yolov4 license plate detector in TensorRT format (trt or engine). And can you share the inference code you used for YOLO inference?

I trained yolov4-tiny from pretrained weights on about 200 labelled license plates it started working okay-ish.
The inference code was exactly the same as in this repo's Demo 5. Train the model, convert to trt and then use trt_yolo.py.
DetectNet_v2 is far worse than yolo for most tasks.


Automated cherry pick of #2495: appsrv: 反转义path segs

Cherry pick of #2495 on release/2.10.0.
#2495: appsrv: 反转义path segs

/lgtm

/approve


Validation object for Picker having Item Source

I want to set the validation object for the picker having item source

I dont get it. Can you show me what you are trying to do?

I dont get it. Can you show me what you are trying to do?

Is it this thread related?

Is it this thread related?

Any news on this @viveklpu2011 ?


Add libc field

{
  "libc": ["glibc"],
  "libc": ["musl"]
}

Use this to specify your package only runs on a specific flavor of libc. This checks against the result from detect-libc.

pnpm support
yarn support
npm support TBD

Thank you


Add files via upload

the university official website URL, if it is different from the domain you are submitting: https://www.stonybrookmedicine.edu/education

a URL of a page on the official website where a long-term (>1 year) IT related course is offered by the university
https://bmi.stonybrookmedicine.edu/courses

a URL of a page or some other proof (.pdf or a screenshot are OK) showing that the university recognizes the domain which you are submitting as an official email domain for the enrolled students.
https://renaissance.stonybrookmedicine.edu/medicine/contact

thanks! have updated based on requirements

@hantswilliams Thank you for your patience while waiting for a review and processing your request. Pull request merged. Please start applying for JetBrains educational subscription at https://www.jetbrains.com/shop/eform/students in an hour, to allow the changes to propagate through JetBrains systems.


ELMo in other languages (Spanish)

ELMo module is available in English.
There are text embedding modules in other languages (basically NNLM in Spanish, German, Chinesse...)
It would be great if we could have ELMo in Spanish and other languages (like ELMoForManyLangs)

@vbardiovskyg , Is it okay, if I work on this issue? or is there someone else working on this already?

Hi Adrish, thanks for the application, we can move the discussion there.

Hi @vbardiovskyg Thanks for the reply.
Is it about GSOC proposal?
Where should we move the discussion to?

Hi Adrish, I meant the GSoC proposal, but I see now that it is not about ELMo. Sorry for confusion.
I don't think anybody is working on this at the moment, so feel free to do so :)

Hi Adrish, I meant the GSoC proposal, but I see now that it is not about ELMo. Sorry for confusion.
I don't think anybody is working on this at the moment, so feel free to do so :)

Hi @vbardiovskyg thanks for the reply. About the GSOC proposal, can you please drop your valuable reviews? Around 10 Hours is left for final submission.

@vbardiovskyg , I'm trying to export the ELMo Graph from torch to ONNX and Then form a TF Graph from it. However, torch.onnx.export says
ONNX export failed: Couldn't export operator aten::sort.
According to Pytorch, There's no support for aten::sort, right now. (https://github.com/pytorch/pytorch/issues/17519#issuecomment-471638973)
Can you suggest some work around?

Hi Adrish, I don't actually have much experience with exporting Pytorch models to ONNX.
If I understand correctly, on the path Pytorch -> ONNX -> TensorFlow, this fails on the first step? Then it would be a question for the ONNX exporting tool project.
A very general answer would be: could you try to understand why that operator is used in the model and whether it could be replaced by something else or dropped? Not sure why one would depend on sorting.

Hi @alberduris  - if you haven't seen this already, we've added many text embedding models in different languages over the years since this comment was left (although ELMo is not one of them) -- you can take a look here & use the left hand filter to select for the languages you'd like: https://tfhub.dev/s?subtype=module,placeholder


Can generate html directly from markdown preview?

I'm using markdown and the preview is very nice, I can see instant preview while typing, but when I use other tool to generate html, it always not as I expect, or I have to adjust markdown source again and again. (Preview is not useful for me anymore)
So does VSCode support to generate html from preview directly? Or this is a new feature request.

I had this question about a year ago: #4481 - and yes, it's a feature request. I haven't kept up with this issue since I raised it, so I have no idea if anything has changed on that front.

We use markdown it for generating the html. Here are instructions on setting up a task to generate html from markdown: https://code.visualstudio.com/Docs/languages/markdown#_compiling-markdown-into-html You still have to bring your own stylesheets.
Exporting the preview html exactly as it appears in the editor is not a scenario we are targeting. Instead, I recommend creating a custom style stylesheet that works for both the preview and the exported html: https://code.visualstudio.com/Docs/languages/markdown#_using-your-own-css


Fix code snippet highlighting

Fix code snippet highlighting in the Built-in middleware section on Using middleware page
Before:

After:

Thanks for catching that @hrmnlt


Documentation on all the infos that can be ripped

A better documentation would be very useful for using this in other projects.

Well, the example.py file covers pretty much everything...from instantiation to "what's available".


OpenBanking

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 0c6e8183-678d-cf30-08d4-47693ea3fc21
Version Independent ID: ec5d9947-7f0d-dad1-6a85-5bc581af9d7f
Content: Push notification templates - PlayFab
Content Source: playfab-docs/features/engagement/push-notifications/push-notification-templates.md
Service: playfab
GitHub Login: @joannaleecy
Microsoft Alias: joanlee

No feedback. Closing.


Help: How to delete these PVC and PV？

Help: How to delete these PVC and PV？
PV and PVC is dynamic-nfs-provisioner create

@GuridMa I see that in the screenshot you hae provided, there is a typo in the command - a missing whitespace when providing -n openebs
I'm not sure if that's indeed the reason, but that's the first I'd ask for to be corrected.

Hi @GuridMa. It's been a while, but if you're still hitting this issue, here's a link to the solution: https://github.com/openebs/openebs/issues/3329
In your outputs above, I don't see the openebs-nfs-provisioner. Is this an issue related to the dynamic-nfs-provisioner?

Closing this issue as it is a known issue, and a duplicate to openebs/openebs#3329.


Demo Storyboard

Load NIRSpec and MIRI cubes

Scrub through each cube
Close them

Load MaNGA cube

Scrub through the cube and leave at H_a (~6840 A)
Keep open

Josh's RGB median (with added color toggling)

Close

Jonathan's ROI (decide where to place before hand)
Show that Jonathan's ROI selected area opens in Nick's SpecView

Fit an emission line (should be same as the one displayed in part 2, H_a)
Display results

Fit another emission line

Display results

Bring up a command line interface in Glue.  Load Ivo's smoothing script.  Smooth the cube in 3D.
Display the smoothed cube (x,y).
Select a region and display the smoothed spectrum.

Well, in reality I don't have any specific script to load. The notebooks use just stuff that is already available at the python command prompt. So in fact the only thing that Glue would be doing is to display the data object that resulted at the end of all that typing in the Glue iPython window.

Some tasks to improve smoothing that ought be done today:

[ ] Merge cube loaders.
[ ] Grab the actual data from subset objects passed through messages (and create Layer object from this).
[ ] Make sure SpectraViewer gets unit information from MaNGA headers (even if not from other data cubes' header).