IPv6 on GitHub-hosted runners

This issue is transferred from here.
In this ticket, the user is trying to reach the IPv6 address of Google by running the command "curl -6 https://www.google.com" on the GitHub-hosted Ubuntu runner, but it always failed to connect and returned the message "Couldn't connect to server".
The command "curl https://www.google.com" or "curl -4 https://www.google.com" can work fine to reach the IPv4 address.
I also tested directly ping the IPv6 address of Google via the command "ping6 ipv6.google.com", it also failed to connect and returned the message "connect: Network is unreachable".
To check whether the current Linux kernel supports IPv6 via the command,
[ -f /proc/net/if_inet6 ] && echo 'IPv6 ready system!' || echo 'No IPv6 support found! Compile the kernel!!'

It returns "IPv6 ready system!", looks like IPv6 is supported.
Then to check whether the IPv6 module has loaded,
lsmod | grep -qw ipv6 && echo "IPv6 kernel driver loaded and configured." || echo "IPv6 not configured and/or driver loaded on the system."

It returns "IPv6 not configured and/or driver loaded on the system.", looks like the IPv6 module is not loaded.
Load the IPv6 module and check again.
modprobe ipv6
lsmod | grep -qw ipv6 && echo "IPv6 kernel driver loaded and configured." || echo "IPv6 not configured and/or driver loaded on the system."

The returned result still shows that the IPv6 module is not loaded. Looks like this should be the reason for failed to reach the IPv6 address.
Why it can't load the IPv6 module on the GitHub-hosted runner? Whether this is an expected behavior for some limitations set on the hosted runners?

@BrightRan looks like the images itself are ok.
@alepauly do we have ipv6 configured on the environment side?

According to the Docker documentation the images would still need to enable IPv6 for Docker containers, even if the VMs start to support it.

Unfortunately we can't support this at this time due to infrastructure constraints.

Unfortunately we can't support this at this time due to infrastructure constraints.

Is it in the roadmap? I found no opening issues on IPv6 support. Thanks.

GitHub IP address list now has IPv6 address for GitHub Actions.
https://api.github.com/meta

GitHub IP address list now has IPv6 address for GitHub Actions.
https://api.github.com/meta

As of a week ago, I could not get IPv6 working in GitHub Actions but let's hope them listing those address's means they are about to release support for it.

As of a week ago, I could not get IPv6 working in GitHub Actions but let's hope them listing those address's means they are about to release support for it.

Tested today.
Under both ubuntu-latest and ubuntu-20.04:
curl -vv --ipv6 "[2607:f8b0:4002:c03::93]:80"
* Rebuilt URL to: [2607:f8b0:4002:c03::93]:80/
*   Trying 2607:f8b0:4002:c03::93...
* TCP_NODELAY set
* Immediate connect fail for 2607:f8b0:4002:c03::93: Network is unreachable
* Closing connection 0
curl: (7) Couldn't connect to server

ip route sh
default via <IP_ADDRESS> dev eth0 proto dhcp src <IP_ADDRESS> metric 100
<IP_ADDRESS>/16 dev eth0 proto kernel scope link src <IP_ADDRESS>
<IP_ADDRESS> via <IP_ADDRESS> dev eth0 proto dhcp src <IP_ADDRESS> metric 100
<IP_ADDRESS> via <IP_ADDRESS> dev eth0 proto dhcp src <IP_ADDRESS> metric 100
<IP_ADDRESS>/16 dev docker0 proto kernel scope link src <IP_ADDRESS> linkdown

ip addr sh
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0d:3a:c6:7b:12 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20d:3aff:fec6:7b12/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:79:06:ee:ce brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global docker0
       valid_lft forever preferred_lft forever

No IPv6 routes so far.

Tested today.
Under both ubuntu-latest and ubuntu-20.04:
curl -vv --ipv6 "[2607:f8b0:4002:c03::93]:80"
* Rebuilt URL to: [2607:f8b0:4002:c03::93]:80/
*   Trying 2607:f8b0:4002:c03::93...
* TCP_NODELAY set
* Immediate connect fail for 2607:f8b0:4002:c03::93: Network is unreachable
* Closing connection 0
curl: (7) Couldn't connect to server

ip route sh
default via <IP_ADDRESS> dev eth0 proto dhcp src <IP_ADDRESS> metric 100
<IP_ADDRESS>/16 dev eth0 proto kernel scope link src <IP_ADDRESS>
<IP_ADDRESS> via <IP_ADDRESS> dev eth0 proto dhcp src <IP_ADDRESS> metric 100
<IP_ADDRESS> via <IP_ADDRESS> dev eth0 proto dhcp src <IP_ADDRESS> metric 100
<IP_ADDRESS>/16 dev docker0 proto kernel scope link src <IP_ADDRESS> linkdown

ip addr sh
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet <IP_ADDRESS>/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:0d:3a:c6:7b:12 brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::20d:3aff:fec6:7b12/64 scope link
       valid_lft forever preferred_lft forever
3: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
    link/ether 02:42:79:06:ee:ce brd ff:ff:ff:ff:ff:ff
    inet <IP_ADDRESS>/16 brd <IP_ADDRESS> scope global docker0
       valid_lft forever preferred_lft forever

No IPv6 routes so far.

Is there any way to get only the IPv4 address of the domain name at the VM level?

@alepauly Time to reopen this! it is 2022 IPv6 is a requirement.
Legacy-v4 might live for a long time (to long) but you can't ignore v6!

Almost the end of January 2022 ... still no IPv6?

I just found out about the non-support for the current Internet Protocol that is now nearly 25 years old.
Github, this is really embarrassing. There really is no excuse to not support that for years.  Pardon my french..

I just found out about the non-support for the current Internet Protocol that is now nearly 25 years old.
Github, this is really embarrassing. There really is no excuse to not support that for years.  Pardon my french..

Summer 2022 and still no IPv6 support. Unacceptable.

@alepauly I don't know why you closed this ticket in the first place but the demand for IPv6 is just getting higher. Do you see any chance of this ever getting fixed or should we all just switch to Gitlab CI or go back to Travis CI.

Any updates on this?

GitHub and the won't fix from @alepauly shows that they have no intention to fixing this. If they did this issue would have remained open.

Unfortunately we can't support this at this time due to infrastructure constraints.

So Github is not able to adapt it's infrastructure in over 2 years. Wow..

Its just IPv6, its not even like they need to change their BGP peers or something.
Same with Azure, they still don't provide IPv6 by default with their services.

Maybe, just maybe there is a little bit of hope on this moving forward. https://github.com/orgs/community/discussions/10539#discussioncomment-3917345

2023 ping - any chance to get runners able to do ipv6? I need it for testing v6 code
For others who might know a workaround: is there a v6 test env easily accessible/compatible with github action syntax?

For others who might know a workaround: is there a v6 test env easily accessible/compatible with github action syntax?

There is always Gitlab. It's better anyway imho.

For others who might know a workaround: is there a v6 test env easily accessible/compatible with github action syntax?

There is always Gitlab. It's better anyway imho. They sadly also don't have IPv6 in their hosted runners but at least you can set it up in your own runners.

I ended up moving some projects to GitLab and now we are able to perform the required testing with IPv6.

Sad!

Please GitHub 🙏

Hi guys, I was trying to add integration test on Hurl a CLI HTTP client based on libcurl.
What does surprise me is that I've been able to make a GET request using curl and ipv6 from a GitHub-hosted macOS runner:
curl on macOS is available under:

/usr/bin/curl 8.1.2
/usr/local/opt/curl/bin/curl 8.3.0

Both binary have IPv6 feature enabled but only curl 8.3.0 is able to GET https://google.com with --ipv6 on GitHub macOS runners.
$ /usr/local/opt/curl/bin/curl --version
curl 8.3.0 (x86_64-apple-darwin21.6.0) libcurl/8.3.0 (SecureTransport) OpenSSL/3.1.2 zlib/1.2.11 brotli/1.1.0 zstd/1.5.5 libidn2/2.3.4 libssh2/1.11.0 nghttp2/1.56.0 librtmp/2.3 OpenLDAP/2.6.6
Release-Date: 2023-09-13
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtmp rtsp scp sftp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS brotli GSS-API HSTS HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe TLS-SRP UnixSockets zstd
$ /usr/bin/curl --version
curl 8.1.2 (x86_64-apple-darwin21.0) libcurl/8.1.2 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.11 nghttp2/1.45.1
Release-Date: 2023-05-30
Protocols: dict file ftp ftps gopher gophers http https imap imaps ldap ldaps mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp
Features: alt-svc AsynchDNS GSS-API HSTS HTTP2 HTTPS-proxy IPv6 Kerberos Largefile libz MultiSSL NTLM NTLM_WB SPNEGO SSL threadsafe UnixSockets

And test with --ipv6 against google.com:
$/usr/local/opt/curl/bin/curl --ipv6 --verbose https://google.com/
...
Connected to google.com (::ffff:<IP_ADDRESS>) port 443
...

$/usr/bin/curl --ipv6 --verbose https://google.com/
curl: (7) Couldn't connect to server

So, it seems there is kind of IPv6 support inside GitHub infra. I'm surprised to see that between curl 8.1.2 and curl 8.3.0 there is a difference of usage with --ipv6 option but I can't explain it.

You'll note however, that's not really IPv6.  It's an IPv4-mapped address.  If it were really IPv6, it would look like this, from my desktop Macbook:
$ curl -V
curl 8.1.2 (x86_64-apple-darwin23.0) libcurl/8.1.2 (SecureTransport) LibreSSL/3.3.6 zlib/1.2.12 nghttp2/1.55.1
[...]
$ curl --ipv6 --verbose https://google.com/
*   Trying [2607:f8b0:4002:c2c::65]:443...
* Connected to google.com (2607:f8b0:4002:c2c::65) port 443 (#0)
[...]

use this
  jobs:
    ipv6:
      runs-on: ubuntu-latest
      steps:
        - name: Setup WARP
          uses: fscarmen/warp-on-actions@v2
        - name: ipv6
          run: |
            curl -m 9 --ipv6 --verbose https://google.com
            curl -m 9 -6 https://ifconfig.co

Here's my workaround (open-source and documented) that I hope that can help you too:

privacy.sexy | force-ipv4: GitHub action to use for GitHub workflows.
privacy.sexy | force-ipv4.sh.: Script for general use.

After days of research and trial/error, this is how I got this working:

Create a script called force-ipv4.sh, that configures system to prefer IPv4 over IPv6, call it to configure the machine. It was not easy to find a reliable cross-platform solution and I went Cloudflare WARP for DNS resolution along with some system configurations.
To easily use the script in GitHub workflows, create GitHub action called force-ipv4 and call it in GitHub runners.
Fixes the IPv6 request issues, and you can happily run e.g. fetch API from Node.

Related commit introducing this fix: https://github.com/undergroundwires/privacy.sexy/commit/52fadcd6177ed06216be9c67dad57192ae02a4f9


Unicode text logging scoops up colours from remote terminals

The change introduced by https://github.com/Fizzadar/pyinfra/commit/11920ab26a0a58f35949df0eb79147e27339c9e0#diff-44ee2c0ef9beeef3f6cf0b8b7553896c + https://github.com/Fizzadar/pyinfra/commit/0a8eb70eb450a13d015422c552b0dc5c9555a8f0#diff-44ee2c0ef9beeef3f6cf0b8b7553896c means than server-side terminal colors/etc are not rendered correctly in the client terminal.
Best option might be to print as-is, and failing that encode the string and print that.

Noope it totally works. Not sure why I thought otherwise 😩


[web] Emojis with skin tone are not rendered appropriately

As shown in the screenshot below:

class MyHomePage extends StatelessWidget {
  final emoji = '👌🏾'; // from MacOS
  final emoji2 = '👋🏾'; // from Twitter
  // final emoji3 = '👌🏾'; // from Twitter throws error: 'Illegal character '55356'.'

  @override
  Widget build(BuildContext context) {
    print('$emoji code Units: ${emoji.codeUnits}  runes: ${emoji.runes}');
    // prints 👌🏾 code Units: [55357, 56396, 55356, 57342]  runes: (128076, 127998)
    print('$emoji2 code Units: ${emoji2.codeUnits}  runes: ${emoji2.runes}');
    // prints 👋🏾 code Units: [55357, 56395, 55356, 57342]  runes: (128075, 127998)

    return Scaffold(
      appBar: AppBar(),
      body: Center(
        child: Text(
          '$emoji \n $emoji2',
          style: TextStyle(fontSize: 24.0),
        ),
      ),
    );
  }
}

This behavior can be observed in Flutter Web, and also can be seen in DartPad (Based on Flutter 2.0.3 Dart SDK 2.12.2).

Most likely related to the discussion in #78422

This is reproducible with CanvasKit only.

code sample
import 'package:flutter/material.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      home: MyHomePage(),
    );
  }
}

class MyHomePage extends StatelessWidget {
  final emoji = '👌🏾'; // from MacOS
  final emoji2 = '👋🏾'; // from Twitter
  // final emoji3 = '👌🏾'; // from Twitter throws error: 'Illegal character '55356'.'

  @override
  Widget build(BuildContext context) {
    print('$emoji code Units: ${emoji.codeUnits}  runes: ${emoji.runes}');
    // prints 👌🏾 code Units: [55357, 56396, 55356, 57342]  runes: (128076, 127998)
    print('$emoji2 code Units: ${emoji2.codeUnits}  runes: ${emoji2.runes}');
    // prints 👋🏾 code Units: [55357, 56395, 55356, 57342]  runes: (128075, 127998)

    return Scaffold(
      appBar: AppBar(),
      body: Center(
        child: Text(
          '$emoji \n $emoji2',
          style: TextStyle(fontSize: 24.0),
        ),
      ),
    );
  }
}

flutter doctor -v
[✓] Flutter (Channel stable, 2.0.4, on macOS 11.2.3 20D91 darwin-x64, locale en-AO)
    • Flutter version 2.0.4 at /Users/pedromassango/Code/flutter_stable
    • Framework revision b1395592de (4 days ago), 2021-04-01 14:25:01 -0700
    • Engine revision 2dce47073a
    • Dart version 2.12.2

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/pedromassango/Library/Android/sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = /Users/pedromassango/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.4, Build version 12D4e
    ! CocoaPods 1.9.3 out of date (1.10.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] IntelliJ IDEA Community Edition (version 2020.3.3)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 54.1.4
    • Dart plugin version 203.7759

[✓] VS Code (version 1.54.3)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.16.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 11.2.3 20D91 darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 89.0.4389.114

! Doctor found issues in 1 category.

[✓] Flutter (Channel master, 2.1.0-13.0.pre.469, on macOS 11.2.3 20D91 darwin-x64, locale en-AO)
    • Flutter version 2.1.0-13.0.pre.469 at /Users/pedromassango/Code/flutter_master
    • Framework revision ccf83d1de9 (2 hours ago), 2021-04-06 12:39:01 +0800
    • Engine revision d5fdd57ce6
    • Dart version 2.13.0 (build 2.13.0-203.0.dev)

[✓] Android toolchain - develop for Android devices (Android SDK version 30.0.2)
    • Android SDK at /Users/pedromassango/Library/Android/sdk
    • Platform android-30, build-tools 30.0.2
    • ANDROID_HOME = /Users/pedromassango/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)
    • All Android licenses accepted.

[!] Xcode - develop for iOS and macOS
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 12.4, Build version 12D4e
    ! CocoaPods 1.9.3 out of date (1.10.0 is recommended).
        CocoaPods is used to retrieve the iOS and macOS platform side's plugin code that responds to your plugin usage on the Dart side.
        Without CocoaPods, plugins will not work on iOS or macOS.
        For more info, see https://flutter.dev/platform-plugins
      To upgrade see https://guides.cocoapods.org/using/getting-started.html#installation for instructions.

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 4.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 1.8.0_242-release-1644-b3-6915495)

[✓] IntelliJ IDEA Community Edition (version 2020.3.3)
    • IntelliJ at /Applications/IntelliJ IDEA CE.app
    • Flutter plugin version 54.1.4
    • Dart plugin version 203.7759

[✓] VS Code (version 1.54.3)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.16.0

[✓] Connected device (2 available)
    • macOS (desktop) • macos  • darwin-x64     • macOS 11.2.3 20D91 darwin-x64
    • Chrome (web)    • chrome • web-javascript • Google Chrome 89.0.4389.114

! Doctor found issues in 1 category.

Current workarounds -

Use the google fonts package

Text(
  '👌🏿',
  style: GoogleFonts.notoColorEmojiCompat(
    color: Colors.black,
    fontSize: 32,
  ),
)

Run using the HTML renderer

flutter run -d chrome --web-renderer html

Current workarounds -

Use the google fonts package

This however, suffers from #76248
Text(
  '👌🏿',
  style: GoogleFonts.notoColorEmojiCompat(
    color: Colors.black,
    fontSize: 32,
  ),
)

OR

Run using the HTML renderer

flutter run -d chrome --web-renderer html

this didn't fix the issue on mobile, iOS


fix: dependent sources compiling

What I did
Updated the discord link to point to the ApeWorX channel
Updated ethpm-types version and pinning
Renamed sources to be more specific about being paths and not actual Source objects.
Added logic for setting source references i.e. dependent sources
Added logic to add source references to needs_compiling

[ ] add TODO comments referencing what will change when breaking update is applied to CompilerAPI
[ ] research what vyper will require
[ ] open another issue for adding type setting/logic to Source objects

fixes: #623
How I did it

How to verify it

Run through the example in the issue/ticket

Checklist

[ ] All changes are completed
[ ] New test cases have been added
[ ] Documentation has been updated

Closing to split up into different tickets #703


Unable to import 4.5G GraphML file

Hi, I get this error when trying to import graphml exported from neo4j.
Error: com.orientechnologies.orient.core.exception.OStorageException
Error: com.orientechnologies.orient.core.exception.OStorageException: Error during removal of mapping between logical and physical record position.
Error: com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.ONestedRollbackException: Atomic operation was rolled back by internal component
There were few errors initially due to heapsize which I got rid of by configuring the script. I am stalled on this one.

@iammayankg Do you have the full stack trace? Usually such internal errors are due to out of memory. How did you configure the JVM? What settings did you use to import it?

Hi, Here's the stacktrace
Importing GRAPHML database from database xxx.graphml...{db=xxx} Error on creating record in cluster: com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster@f5d240d
com.orientechnologies.orient.core.exception.OStorageException
at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.createRecord(OPaginatedCluster.java:444)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.doCreateRecord(OAbstractPaginatedStorage.java:1668)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commitEntry(OAbstractPaginatedStorage.java:2152)
at com.orientechnologies.orient.core.storage.impl.local.OAbstractPaginatedStorage.commit(OAbstractPaginatedStorage.java:1016)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.doCommit(OTransactionOptimistic.java:596)
at com.orientechnologies.orient.core.tx.OTransactionOptimistic.commit(OTransactionOptimistic.java:148)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2549)
at com.orientechnologies.orient.core.db.document.ODatabaseDocumentTx.commit(ODatabaseDocumentTx.java:2518)
at com.tinkerpop.blueprints.impls.orient.OrientTransactionalGraph.commit(OrientTransactionalGraph.java:161)
at com.orientechnologies.orient.graph.console.OGraphMLReader.inputGraph(OGraphMLReader.java:297)
at com.orientechnologies.orient.graph.console.OGraphMLReader.inputGraph(OGraphMLReader.java:118)
at com.orientechnologies.orient.graph.console.OGraphMLReader.inputGraph(OGraphMLReader.java:93)
at com.orientechnologies.orient.graph.console.OGremlinConsole.importDatabase(OGremlinConsole.java:119)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.orientechnologies.common.console.OConsoleApplication.execute(OConsoleApplication.java:382)
at com.orientechnologies.common.console.OConsoleApplication.executeCommands(OConsoleApplication.java:247)
at com.orientechnologies.common.console.OConsoleApplication.run(OConsoleApplication.java:136)
at com.orientechnologies.orient.graph.console.OGremlinConsole.main(OGremlinConsole.java:65)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.ArrayDeque.(Unknown Source)
at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OWALChangesTree.applyChanges(OWALChangesTree.java:248)
at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OWALChangesTree.getLongValue(OWALChangesTree.java:118)
at com.orientechnologies.orient.core.storage.impl.local.paginated.wal.OWALChangesTree$PointerWrapper.getLong(OWALChangesTree.java:788)
at com.orientechnologies.common.serialization.types.OLongSerializer.deserializeFromDirectMemory(OLongSerializer.java:130)
at com.orientechnologies.orient.core.storage.impl.local.paginated.base.ODurablePage.getLongValue(ODurablePage.java:108)
at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedClusterState.getFreeListPage(OPaginatedClusterState.java:65)
at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.findFreePage(OPaginatedCluster.java:1528)
at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.addEntry(OPaginatedCluster.java:1476)
at com.orientechnologies.orient.core.storage.impl.local.paginated.OPaginatedCluster.createRecord(OPaginatedCluster.java:433)
... 20 more
Error: com.orientechnologies.orient.core.exception.OStorageException
Error: com.orientechnologies.orient.core.exception.OStorageException: Error during removal of mapping between logical and physical record position.
Error: com.orientechnologies.orient.core.storage.impl.local.paginated.atomicoperations.ONestedRollbackException: Atomic operation was rolled back by internal component

In server.sh I configured:
MAXHEAP=-Xmx1G
and
MAXDISKCACHE="-Dstorage.diskcache.buffersize=5200"
I imported it using the console command "Import graphml". Initially it threw a java lang memory/GC error, but after setting the configuration as mentioned on the OrientDB performance page those went away.

How much RAM do you have in that server? Could you also try setting -Dstorage.useWAL=false ?

Hi, The server has about 8G of free RAM.
I tried disabling WAL and got the following error:
Importing GRAPHML database from database xxx.graphml...
Error: com.orientechnologies.orient.core.exception.OStorageException: WAL mode is not active. Transactions are not supported in given mode
If there is any recommended setting for the server I can consider that. My data is about 4.5G in GraphML format.

I've introduced a new param in importer: batchSize. Please use last 2.1-SNAPSHOT and take a look at: http://orientdb.com/docs/last/Import-from-Neo4j-into-OrientDB.html

Hi @Ivca, I am running in the same problem. Testing Orientdb importer: IMPORT DATABASE /tmp/out.graphml batchSize=100 storeVertexIds=true :
Importing GRAPHML database from /tmp/out.graphml with options (batchSize=100 storeVertexIds=true)...
Error: java.lang.OutOfMemoryError: GC overhead limit exceeded using version 2.1.0

How much RAM are you using or OrientDB in terms of heap? How your vertices/edges are big? Have you tried to reduce the batchSize to 20 ?

I will try reduce batch to 20; exported db is 16GB (in neo is 8GB? maybe another format). Where do I look for the RAM assigned in heap ? couldn't find in config/orientdb-server-config.xml

edit oetl.sh and change -Xmx512m to -Xmx2G

nope, did not work out. maybe some issue in the db exported from neo?
I would like to taste the difference, especially in lucene queries (find meaningful results to start a traversal). Maybe I should import the db from csv..

If you can send us your database, we can try importing it.

it s 16 gb @lvca , i m trying with pyorient to reimport it.  it is an undirected graph, i also posted some questions about undirected graph on google discussion, might you have some time to clarify model to adopt respect the query i d like to use.

hi @gg4u, did you resolve this issue using  pyOrient?

Closed because no response.


feat(normalization): adiciona função que remove caracteres especiais

Coverage remained the same at 100.0% when pulling 68e05199c978f807ff5b576955349e14e7475224 on develop into 23bc27f2d07dad09927187aed2f9e3461eec05b4 on master.


Attend and present intake onboarding

Describe the Issue
There are several presentations to new Ministry Team clients on the afternoon of Oct 26.  We like to have different presenters because different voices and faces make it more interesting for the audience.
Additional Context

See the Calendar invite for Oct 26 1pm for details

Acceptance Criteria

Co-present either Emily B's Policy deck if she cannot present (get Julian to update the presenter intro slides for Catherine if we do this) OR
Assist Julian in presenting the intro deck.  Will need to practice once and insert the "who's speaking next" powerpoint breadcrumb hints that we use to smoothly transfer from one speaker to the next.

.
.
.
.
.
.
.
.
.
.
.
Extra Template Info
Definition of Ready
These set of conditions will need to be met in order to bring a ticket into the sprint and start work. Protects the team from unclear requirements.
Issues (Task/Story/Spike) aligned to an EPIC and linked appropriately.
Assigned to the appropriate CPF MEMBER (and is not left blank in ZenHub) at the start of sprint (1st day of the sprint)
Acceptance has been defined for the issue and has been reviewed with CPF team and approved by the necessary approver.
Has been sized and estimated by the delivery team.
Detailed breakdown of the steps required to complete the story in the additional context
Any additional specifications have been documented, reviewed with CPF and approved by the necessary approver.
Acceptance Criteria
A set of pre-defined requirement that need to be met in order to mark the user story as “done”.
It should be testable with no room for interpretation
It should be either “pass” or “fail”
It should be clear enough for business stake holders to understand
As part of the user story, it should be written from the user perspective
A well written acceptance criteria is great for

managing expectations
defining scope
reducing ambiguity
establishing testing criteria for QA
Given (Pre-condition)
When (Action)
Then (Outcome)

Definition of Done
These set of conditions are met for work to be considered as complete on an issue.
All acceptance criteria(s) have been validated.
All the necessary documentation for the issue has been uploaded to Teams.
Functionality has been tested when applicable.
Functionality has been demonstrated to the relevant stakeholders (where applicable).
Story has been scheduled for a Sprint Demo/community update and impacted teams invited to the Demo.
Stories accepted by PO and documented for potential sharing with impacted users.

Documented = Captured in Community Update (CU) deck
Release notes = aspirational goal of providing a bulleted list of features and changes that our users can touch. Was going to store them in our Cloud Pathfinder repo. Can also display to users in the CU

Reviewing the video record in Teams.

Attended and assisted Julian in presenting the intro deck. Ticket can be closed.


Add timezone and date_format option

Due to CSV parser does not quote comma like ",", embulk-output-bigquery need to use this formater.
However, this plugin need to modify timestamp format. BQ can't use timezone +9:00.
See also Twitter timeline Japanese.

Thanks your PullReq! :bow:


[Research] Try to reduce permissions to access to URLs

Follow-up for #1
Current set-up (v. 1.0.0) is better than <all_urls>, but it still requests access for all the pages contents on all the Google search domains.
To check: if it's possible to implement it by only accessing URLs.
I think that some drawbacks will come from such an implementation though, e.g.:

go to Google search results page
copy a link address

In that case the version with access to contents would replace the link's URL before the copying and the copied URL wouldn't contain the text fragment section. However if the URL is changed during navigation, copying operation would still give the unmodified URL.

The issue will be closed due to the aforementioned unsolved issue. It might be reopened in the future if necessary.


Option for not hiding player details when player has no offer

I like to have the player details always visible. So an option to not hide them would be awesome.

Hi @eLindros ,
I added a new setting to change the behavior. You can set it in the settings view:

You can try it with the current release candidate of version 1.6:
https://kickbaseplus.fabian-fischer.com/
Let me know what you think.
Fabian

Thanks! Exactly was I was looking for.

@all-contributors please add @eLindros for ideas and code

Hi @eLindros  ,
I released it with version 1.6 (#58).
please take a look or just test it here: https://kickbaseplus.fabian-fischer.com/
I will close this ticket, but if you have any further ideas, just contact me or open a new issue.
best,
Fabian


operationTimeout and connectionTimeout functionality

In lib/connect-couchbase.js we see operationTimeout and connectionTimeout being added to connectionOptions but neither are explicitly used. In fact on line 114, it looks like connectionTimeout is defaulted to 10000 regardless of what was passed into the configuration.
I haven't been able to get either feature working as the README explains and I believe this may be the reason. Any insight would be greatly appreciated!

Hi,
The reason for this was in the updated version of Couchnode, the ability to set connection timeout was removed from the public API.  I expect they will add it back, or maybe have since added it back.
Feel free to submit a pull request if the latter is the case, or I will look into it as soon as I can.
Thanks.
Chris

I think I put that hardcoded 10000 in there during testing and forgot to remove it.  I was testing whether or not the new couchnode API responded to setting "client.connectionTimeout = xxx".

Update, I see connectionTimeout is part of the API.  I'll make sure that gets added back and I'll have NPM update published soon.
NOte that I have removed the dependency on Couchbase from the connect-couchbase library.  It is up to the developer to ensure Couchbase 2.x is installed via NPM and is available.

This has been fixed and NPM updated:  https://github.com/christophermina/connect-couchbase/commit/446ecc3f705a25fc94ab0e23a28c4aa4a5c99626


Adding task functionality does not work.

Clicking the submit button after entering information for a new task does not add a new task to the list of tasks.

Fixed in the latest commit. 253a29dd49fb8a48df78e0cbdda2288d14f86520


[Pie Chart] New feature: added a badge widget in addition to title

I needed a feature to add some kind of creative freedom in creating the title. So I added a way to add a widget to represent the title of the section. However, you can still have the default title as an option in addition to this widget. Having it as a widget also makes it intuitive to create and easy for novice programmers.

Main Changes:

PieChart:

added child widget to CustomPaint to render the badge widgets
made default duration static to reuse externally for consistency

PieChartSectionData:

title: changed default to value
added params for badge widget

PieChartPainter:

added function to provide position offsets to parent
renamed _drawTexts to _calculateOverlays to better represent the various functions

Misc:

Added typedef for position offsets map: GetPositionOffsetsFunction
Added test for PieChart's badge widget
Added sample source code for pie chart with badge widgets
Updated documentation

Cool!

Hi dude, I am sorry for the delay.
I have just changed this PR's target branch to dev.
Today we have made some changes on dev, Now there is a conflict.
Can you please fix it?
Thanks, I will review it ASAP.

Hey, thanks for the reponse 😄
I have fixed the conflict. Looking forward to your review. 🙏 😇

@gmaggio Thanks for the PR. It is a huge contribution to my project also. I am waiting for the merge too. Looks like the conflicts are resolved. @imaNNeoFighT no pressure but you would make me happy too if you have time to test and possibly merge this :)

I'm on it :)

Hey @gmaggio.

I appreciate your great solution and great work, I bet you it helps people a lot. And they are going to be happy.
You did a great job!
There is only one minor suggestion from me. Now we don't need the title property in PieChartSectionData anymore and we can remove it. Because we can have a Text widget instead.
We can define a Text widget as a default value for the badge widget and show the index of each section inside it.
How do you think?

Oh, I see you have used both of them at the same time.
Now you can skip my suggestion, but I would like to hear your idea.
Thanks!

I think we can keep title to prevent make a breaking-change.
If you think we don't need it anymore, we can mark it deprecated!
I need your idea.
Thanks!

Hi @imaNNeoFighT. Thank you for the complement. I really appreciate you looking into my contribution. 😇
In regards to the title, I think we should just keep it for a couple of reasons:

It uses CustomPainter, which paints directly on the canvas during the painting phase.
Though, to be honest, I'm not well versed on canvas painting, so my current assumption is that it is better than widget manipulation it terms of performance. Correct me if I'm wrong. 🤔
The badge widget has yet to be tested in various use cases and scenarios.
I had only tried it for my personal needs and it is working okay so far. But we need to allow time for others to use it as well for better feedback. Let's hope it is working as intended, though. 😁
Keeping the title gives users more options.
The title is technically more intuitive since it's more straight forward to use. Most people should be sufficient with this. The badge widget, on the other hand, allows customisation to the regular title but with an added effort, especially if animation is to be used. So we can leave the decision to the users to use whichever is suitable for their needs. And, of course, you can even use both of them at the same time, as you already know. 😉

These are only my point of views, though. Let me know what you think.
Cheers!

All of your points make sense, and we will have both.
Thank you again!


Multiple Test Failures in Master

I've identified multiple tests failures in the master branch.

Fixed all test failures except rv64uc-p-rvc with a simple patch.

https://github.com/sifive/ProcKami/pull/307


/Pods/SFaceCompare/SFaceCompare/SameFace.framework/SameFace' does not contain bitcode.

Hi,
When I try to build for my iPhone X device I get this error:
'/Users/////////Pods/SFaceCompare/SFaceCompare/SameFace.framework/SameFace' does not contain bitcode. You must rebuild it with bitcode enabled (Xcode setting ENABLE_BITCODE), obtain an updated library from the vendor, or disable bitcode for this target. file '/Users////////*/Pods/SFaceCompare/SFaceCompare/SameFace.framework/SameFace' for architecture arm64
I disabled bitcode by going to your target's Build Settings -> Set Enable Bitcode to "No" for the SFaceCompare, but the issue continue.
I'm using: XCode Version 11.4 (11E146)
If more information is needed, just let me know.
Thanks,

Hi @hsabino
I've fixed the problem, just waiting for the pull request to be accepted. We'll hopefully soon find an updated version on CocoaPods. In the meantime you can use SameFace.framework from my pull request which will fix your issues.

@horrion I've just updated and rebuild SameFace.framework. For new updates. Probably your help may require. Cause building SameFace.framework universal framework is a pain for me) And anything may go wrong.
Could you try the new version and let me know, pls.

Wait, so you've built the framework, but not as a universal framework and now you'd like me to build it as a universal framework?


Update kml service; rename to 3d instead of 3D.

@anttikajanus -Can you take a quick look. Just swapped out the service and did a minor rename.

@katydalton looks good.


Write Script to Generate Wiki Boilerplate

At this point, there's A LOT of boilerplate in our wiki, and I'd like to automate that somehow.
I'm a huge fan of writing a script for this because we could eventually add even more features to it. How cool is that?!

Perhaps we can create a tools folder and drop the script in there.

It depends on what exact tasks and how you want to automate them.
I would be happy to help as I like automating things (and said I'll have a look at this repo on dev.to).

I want to take a stab at "automating" the wiki. I'm interested in trying to replicate what we already have through a script, so someone could theoretically just run the script as needed and commit the changes. It might even be nice to automate the wiki building through Jenkins or something, but I'm happy with a working script that we have to manually trigger for now.

Another (although still new in GH) would be to have a bot (say a probot) that will do that or at least the legwork and leave the review/confirmation bit to contributors (like you have with PR checkers).

Wow, that's really cool. I like that idea. I'd be interested in learning TypeScript just to fool around with one of those bots.

For the moment, I'm going to toy around with generating a few markdown pages in Python. If that ends up being a waste of time, I'll look for some help with the probot idea.

I ended up merging in a simple Python script that does some basic analysis of the repo. Unfortunately, it doesn't even scratch the surface in terms of what I'd like to do.
For instance, I'd like to be able to document a list of issues associated with the various languages as well as their contributors. All the analytics I've done so far is bonus, honestly. The GitHub related stuff is actually practical but more complicated in terms of local scripting. Perhaps the probot is the way to go.

I'd like to be able to document a list of issues associated with the various languages as well as their contributors

Was that using labels or keywords in the titles associated to each languages?

Language keywords. That would probably be too many labels otherwise.

Ah I see.

I think I can automate that actually. Instead of linking individual issues, we can link a query string that looks for the language keyword. That should be solid enough!

That looks good to me.

As a summary, the script seems to work great! I'd love to clean it up before adding new features, but for now it does the job.


Revert "haskellPackages: update stackage and hackage"

Reverts NixOS/nixpkgs#351154
Let's move the merge to staging-next to make it somewhat easier on the infra.  This has been discussed a bit in #staging chat.

Reapplied on staging-next.


Remove unnecessary include EGL/eglmesaext.h

This header is not apparently necessary, and it is not present in non-mesa SDKs.

BTW, this include is preventing the backend from building on i.MX6 platforms with Vivante drivers.


Azure AD Password Protection and PCNS

Are there any conflicts between Azure AD Password Protection and PCNS (password change notification service) - part of FIM/MIM to sync passwords across domains?  I presume the password filtering occurs first and after the password is validated by the filters as valid, it is handed off to the PCNS service and the NTDS service for processing (forwarding to MIM/storing in AD)?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: bd810b22-4e2a-5a66-db90-dd39b1b5c565
Version Independent ID: 8e36df09-f7f1-dcbd-dc3d-93cbeed1b040
Content: On-premises password protection FAQ - Azure Active Directory
Content Source: articles/active-directory/authentication/howto-password-ban-bad-on-premises-faq.md
Service: active-directory
Sub-service: authentication
GitHub Login: @iainfoulds
Microsoft Alias: iainfou

@Ken-Br
Thanks for your feedback! We will investigate and update as appropriate.

Hi @Ken-Br,
There should not be any conflicts between AADPP and PCNS.  Windows\ActiveDirectory will not accept\persist a new password unless\until all registered password filter dlls (of which AADPP is one) have accepted the new password.  Upon acceptance, all password filter dlls get notified (see https://docs.microsoft.com/windows/win32/api/ntsecapi/nc-ntsecapi-psam_password_notification_routine) which I presume is the point at which PCNS will then sync the password.   If PCNS syncs the password prior to the notify routine being called, that would be a bad bug in PCNS which is unlikely.
Does this help?
thanks,
Jay

Yes it does - thank you.  Just wanted to be sure because we have many man-years of work that depends upon PCNS working to sync passwords across domains.


Implement Scaling Provider Factory Method Pattern

This commit introduces a factory method pattern and a new package,
cloud that implements all cloud provider specific functionality
for scaling operations. This model will pave the way for supporting
multiple cloud providers and possibly other more generic scaling
backends.
This change moves the worker pool state tracking object into the
worker pool object and allows us to track the desired scaling
direction to simplify the number of objects that need to be
passed around between the core and the providers.
We also introduce a new state field that allows the core to register
nodes as eligible for targeted scaling operations. At present, this
is used to specify nodes that have been identified as low utilization,
have been placed in drain mode and are eligible for detachment and
termination.
The core cluster scaling methods have been updated to use very
fine-grained locking and no longer block the node discovery thread
from updating the node registry while cluster scaling operations are
underway. This allows us to monitor the node registry to confirm a
newly launched instance has succesfully joined the cluster instead
of explicitly calling Nomad API endpoints.
Finally, this commit introduces tracking of node registration
timestamp information to allow methods to understand in what order
nodes within a worker pool were registered.
This PR is a rebased version of #181.
Closes #180

Cluster Scale-Out Event:
[INFO] command/agent: running version 1.1.0-dev
[INFO] command/agent: starting replicator agent...
[INFO] command/agent: replicator is running with cluster scaling globally enabled
[INFO] command/agent: replicator is running with job scaling globally disabled
[WARNING] client/node_discovery: node 91563960-6bbc-b1e0-06df-0b0cd196ebdd is down, initiating deregistration of the node
[WARNING] client/node_discovery: node aef7ecda-8913-1b8d-764f-f862819fad4d is down, initiating deregistration of the node
[INFO] client/node_discovery: worker pool container-node-public-nonprod has 2 healthy nodes configured for scaling: [fcbc41b5-f788-1364-430d-cb0c289303db a30195c3-27cb-91d9-5691-26ef77e06d80]
[INFO] client/consul: leadership lock successfully obtained at replicator/config/leader
[INFO] client/job_scaling_policies: added new policy for job ceapp and group ceapp
[INFO] core/cluster_scaling: worker pool container-node-public-nonprod requires a scaling operation: (Direction: Out, Nodes: 2, Metric: Memory, Capacity: 984, Utilization: 840, Max Allowed: 432)
[INFO] provider/aws: initiating cluster scale-out operation for worker pool container-node-public-nonprod
[INFO] provider/aws: attempting to verify the autoscaling group scaling operation for worker pool container-node-public-nonprod has completed successfully
[INFO] provider/aws: verified the autoscaling operation for worker pool container-node-public-nonprod has completed successfully
[INFO] provider/aws: determining most recently launched instance in autoscaling group container-node-public-nonprod
[INFO] provider/aws: instance i-0081cb2d91a0d9d3b is the newest instance in autoscaling group container-node-public-nonprod and was launched after the threshold 2017-09-23 14:47:13.332317139 +0000 UTC m=+34.333968521
[INFO] core/helper: searching for a registered node with address <IP_ADDRESS> in worker pool container-node-public-nonprod
[INFO] client/node_discovery: worker pool container-node-public-nonprod has 3 healthy nodes configured for scaling: [fcbc41b5-f788-1364-430d-cb0c289303db a30195c3-27cb-91d9-5691-26ef77e06d80 6c9a68a3-5b20-fe68-2ef2-50e03e883312]
[INFO] core/helper: node 6c9a68a3-5b20-fe68-2ef2-50e03e883312 was found as a registered and healhty node with address <IP_ADDRESS> in worker pool container-node-public-nonprod

Alright, this should be ready to go for an initial pass of the cloud provider model. I will be filing several enhancements and changes once this has been merged in.

@jrasell updated to address your review


only 4 threads are used

LocalAI version:
latest docker image
Environment, CPU architecture, OS, and Version:
Ryzen 9 3900X -> 12 Cores 24 Threads
windows 10 -> wsl (<IP_ADDRESS>-microsoft-standard-WSL2 ) docker
Describe the bug
i have the model ggml-gpt4all-l13b-snoozy.bin
but only a maximum of 4 threads are used.
docker-compose.yml
version: "3.9"
services:
  localai:
    image: quay.io/go-skynet/local-ai:latest
    volumes:
      - ./models:/models
      - /etc/localtime:/etc/localtime:ro
    ports:
      - "8080:8080"
    command:
       --models-path /models
       --context-size 1024
       --threads 23
       --debug

model file
models/gpt-3.5-turbo.yaml
name: gpt-3.5-turbo
# Default model parameters
parameters:
  # Relative to the models path
  model: ggml-gpt4all-l13b-snoozy.bin
  # temperature
  temperature: 0.3
  # all the OpenAI request options here..

# Default context size
context_size: 512
threads: 23
# Define a backend (optional). By default it will try to guess the backend the first time the model is interacted with.
#backend: gptj # available: llama, stablelm, gpt2, gptj rwkv
# stopwords (if supported by the backend)
stopwords:
- "HUMAN:"
- "### Response:"
# define chat roles
roles:
  user: "HUMAN:"
  system: "GPT:"
template:
  # template file ".tmpl" with the prompt template to use by default on the endpoint call. Note there is no extension in the files
  completion: completion
  chat: ggml-gpt4all-l13b-snoozy

template
The prompt below is a question to answer, a task to complete, or a conversation to respond to; decide which and write an appropriate response.
### Prompt:
{{.Input}}
### Response:

e.g. command
curl http://localhost:8080/v1/chat/completions -H "Content-Type: application/json" -d '{
     "model": "ggml-gpt4all-l13b-snoozy.bin",
     "messages": [{"role": "user", "content": "What is Kubernetes?"}],
     "temperature": 0.7
   }'

thanks

good catch, thanks for filing up an issue!
that looks a regression, somehow the patch got lost when upstreaming the binding. I've opened up a PR upstream too: https://github.com/nomic-ai/gpt4all/pull/836

should your branch itself be executable?
I have built a new version with it, but unfortunately comes an error that he can not load the model
localai_1  | 9:00AM DBG Request received: {"model":"gpt-3.5-turbo","file":"","language":"","response_format":"","size":"","prompt":null,"instruction":"","input":null,"stop":null,"messages":[{"role":"system","content":"You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown."},{"role":"user","content":"hey"}],"stream":true,"echo":false,"top_p":0,"top_k":0,"temperature":0.5,"max_tokens":1000,"n":0,"batch":0,"f16":false,"ignore_eos":false,"repeat_penalty":0,"n_keep":0,"mirostat_eta":0,"mirostat_tau":0,"mirostat":0,"seed":0,"mode":0,"step":0}
localai_1  | 9:00AM DBG Parameter Config: &{OpenAIRequest:{Model:ggml-gpt4all-l13b-snoozy.bin File: Language: ResponseFormat: Size: Prompt:<nil> Instruction: Input:<nil> Stop:<nil> Messages:[] Stream:false Echo:false TopP:0 TopK:0 Temperature:0.5 Maxtokens:1000 N:0 Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 Seed:0 Mode:0 Step:0} Name:gpt-3.5-turbo StopWords:[HUMAN: ### Response:] Cutstrings:[] TrimSpace:[] ContextSize:1024 F16:false Threads:20 Debug:true Roles:map[system:GPT: user:HUMAN:] Embeddings:false Backend: TemplateConfig:{Completion:completion Chat:ggml-gpt4all-l13b-snoozy Edit:} MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 ImageGenerationAssets: PromptCachePath: PromptCacheAll:false PromptStrings:[] InputStrings:[] InputToken:[]}
localai_1  | 9:00AM DBG Stream request received
localai_1  | 9:00AM DBG Template found, input modified to: The prompt below is a question to answer, a task to complete, or a conversation to respond to; decide which and write an appropriate response.
localai_1  | ### Prompt:
localai_1  | GPT: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.
localai_1  | HUMAN: hey
localai_1  | ### Response:
localai_1  |
localai_1  | [<IP_ADDRESS>]:62236  200  -  POST     /v1/chat/completions
localai_1  | 9:00AM DBG Loading model 'ggml-gpt4all-l13b-snoozy.bin' greedly
localai_1  | 9:00AM DBG [llama] Attempting to load
localai_1  | 9:00AM DBG Loading model llama from ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG Loading model in memory from file: /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | llama.cpp: loading model from /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG Sending chunk: {"object":"chat.completion.chunk","model":"gpt-3.5-turbo","choices":[{"delta":{"role":"assistant"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
localai_1  |
localai_1  | error loading model: llama.cpp: tensor '�~��T��zuƝW�z���$�;�GY�������>\��uƷm�;̈́ȅhLc������m��I��;�]V��zWkxȅzT\�m
                                                                                <���wn��wkZ֨˺�K۶<�ukz��ww����wvV��m
                                                                                <���騳V�E�����Y�۶�;��K�s���V�gV���m<wih���x�vL�X��J��m�;|�?Y_��Ew���Df(ܒ$�;����l��X�yQ�w�)' should not be<PHONE_NUMBER>-dimensional
localai_1  | llama_init_from_file: failed to load model
localai_1  | 9:00AM DBG [llama] Fails: failed loading model
localai_1  | 9:00AM DBG [gpt4all] Attempting to load
localai_1  | 9:00AM DBG Loading model gpt4all from ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG Loading model in memory from file: /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | llama.cpp: loading model from /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | error loading model: llama.cpp: tensor '�~��T��zuƝW�z���$�;�GY�������>\��uƷm�;̈́ȅhLc������m��I��;�]V��zWkxȅzT\�m
                                                                                <���wn��wkZ֨˺�K۶<�ukz��ww����wvV��m
                                                                                <���騳V�E�����Y�۶�;��K�s���V�gV���m<wih���x�vL�X��J��m�;|�?Y_��Ew���Df(ܒ$�;����l��X�yQ�w�)' should not be<PHONE_NUMBER>-dimensional
localai_1  | llama_init_from_file: failed to load model
localai_1  | LLAMA ERROR: failed to load model from /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG [gpt4all] Fails: failed loading model
localai_1  | 9:00AM DBG [gptneox] Attempting to load
localai_1  | 9:00AM DBG Loading model gptneox from ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG Loading model in memory from file: /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | gpt_neox_model_load: invalid model file '/models/ggml-gpt4all-l13b-snoozy.bin' (bad magic)
localai_1  | gpt_neox_bootstrap: failed to load model from '/models/ggml-gpt4all-l13b-snoozy.bin

should your branch itself be executable?
I have built a new version with it, but unfortunately comes an error that he can not load the model
localai_1  | 9:00AM DBG Request received: {"model":"gpt-3.5-turbo","file":"","language":"","response_format":"","size":"","prompt":null,"instruction":"","input":null,"stop":null,"messages":[{"role":"system","content":"You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown."},{"role":"user","content":"hey"}],"stream":true,"echo":false,"top_p":0,"top_k":0,"temperature":0.5,"max_tokens":1000,"n":0,"batch":0,"f16":false,"ignore_eos":false,"repeat_penalty":0,"n_keep":0,"mirostat_eta":0,"mirostat_tau":0,"mirostat":0,"seed":0,"mode":0,"step":0}
localai_1  | 9:00AM DBG Parameter Config: &{OpenAIRequest:{Model:ggml-gpt4all-l13b-snoozy.bin File: Language: ResponseFormat: Size: Prompt:<nil> Instruction: Input:<nil> Stop:<nil> Messages:[] Stream:false Echo:false TopP:0 TopK:0 Temperature:0.5 Maxtokens:1000 N:0 Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 Seed:0 Mode:0 Step:0} Name:gpt-3.5-turbo StopWords:[HUMAN: ### Response:] Cutstrings:[] TrimSpace:[] ContextSize:1024 F16:false Threads:20 Debug:true Roles:map[system:GPT: user:HUMAN:] Embeddings:false Backend: TemplateConfig:{Completion:completion Chat:ggml-gpt4all-l13b-snoozy Edit:} MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 ImageGenerationAssets: PromptCachePath: PromptCacheAll:false PromptStrings:[] InputStrings:[] InputToken:[]}
localai_1  | 9:00AM DBG Stream request received
localai_1  | 9:00AM DBG Template found, input modified to: The prompt below is a question to answer, a task to complete, or a conversation to respond to; decide which and write an appropriate response.
localai_1  | ### Prompt:
localai_1  | GPT: You are ChatGPT, a large language model trained by OpenAI. Follow the user's instructions carefully. Respond using markdown.
localai_1  | HUMAN: hey
localai_1  | ### Response:
localai_1  |
localai_1  | [<IP_ADDRESS>]:62236  200  -  POST     /v1/chat/completions
localai_1  | 9:00AM DBG Loading model 'ggml-gpt4all-l13b-snoozy.bin' greedly
localai_1  | 9:00AM DBG [llama] Attempting to load
localai_1  | 9:00AM DBG Loading model llama from ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG Loading model in memory from file: /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | llama.cpp: loading model from /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG Sending chunk: {"object":"chat.completion.chunk","model":"gpt-3.5-turbo","choices":[{"delta":{"role":"assistant"}}],"usage":{"prompt_tokens":0,"completion_tokens":0,"total_tokens":0}}
localai_1  |
localai_1  | error loading model: llama.cpp: tensor '�~��T��zuƝW�z���$�;�GY�������>\��uƷm�;̈́ȅhLc������m��I��;�]V��zWkxȅzT\�m
                                                                                <���wn��wkZ֨˺�K۶<�ukz��ww����wvV��m
                                                                                <���騳V�E�����Y�۶�;��K�s���V�gV���m<wih���x�vL�X��J��m�;|�?Y_��Ew���Df(ܒ$�;����l��X�yQ�w�)' should not be<PHONE_NUMBER>-dimensional
localai_1  | llama_init_from_file: failed to load model
localai_1  | 9:00AM DBG [llama] Fails: failed loading model
localai_1  | 9:00AM DBG [gpt4all] Attempting to load
localai_1  | 9:00AM DBG Loading model gpt4all from ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG Loading model in memory from file: /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | llama.cpp: loading model from /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | error loading model: llama.cpp: tensor '�~��T��zuƝW�z���$�;�GY�������>\��uƷm�;̈́ȅhLc������m��I��;�]V��zWkxȅzT\�m
                                                                                <���wn��wkZ֨˺�K۶<�ukz��ww����wvV��m
                                                                                <���騳V�E�����Y�۶�;��K�s���V�gV���m<wih���x�vL�X��J��m�;|�?Y_��Ew���Df(ܒ$�;����l��X�yQ�w�)' should not be<PHONE_NUMBER>-dimensional
localai_1  | llama_init_from_file: failed to load model
localai_1  | LLAMA ERROR: failed to load model from /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG [gpt4all] Fails: failed loading model
localai_1  | 9:00AM DBG [gptneox] Attempting to load
localai_1  | 9:00AM DBG Loading model gptneox from ggml-gpt4all-l13b-snoozy.bin
localai_1  | 9:00AM DBG Loading model in memory from file: /models/ggml-gpt4all-l13b-snoozy.bin
localai_1  | gpt_neox_model_load: invalid model file '/models/ggml-gpt4all-l13b-snoozy.bin' (bad magic)
localai_1  | gpt_neox_bootstrap: failed to load model from '/models/ggml-gpt4all-l13b-snoozy.bin

thanks for the heads up, fix for this issue is in https://github.com/go-skynet/LocalAI/pull/507

thank you, the changes work for me.
but unfortunately the output is still extremely slow.
if i run this on the same device with the same model and the software from gpt4all, there is an instant response (3seconds), with localai 1minutes, this is a huge difference

You shouldn't overbook threads, but rather match the number of physical cores. Here I get 120ms per token, but my hardware ain't much capable either

I have also tried it with 12 cores (actual number of pyhsic cores), but it is the same resultis about 1 minute, until even the first word arrives

here with 8threads ggml-gpt4all-l13b-snoozy.bin takes about 15s to reply to "How are you?" (note meanwhile my CPU was busy doing other things):

also note that the model will be loaded into memory at the first usage, so it will be slightly slower on the first call.

i run it via docker, something seems to go wrong it doesn't look like anything is loaded into memory.
when i make a request, nothing really happens for 1 minute, and then the first tokens arrive

I get the same tensor error


Use "full" localization ids in the AltText class (PR 18674 follow-up)

Apparently I missed these in PR #18674.

/botio integrationtest


Karlburke/indexing efficiencies round2

Clean up remaining ontology-api references now that display_subtype is calculated in entity-api rather than HuBMAP's search-api.

@maxsibilla @libpitt @tjmadonna can you help confirm that this change doesn't affect the SenNet search-api?

@yuanzhou I don't think we need self.ONTOLOGY_API_BASE_URL anymore. In the main of the SenNet Search API we initialize our connection to UBKG and pass that instance to the Search Adaptor SearchAPI class


Bump support up to react-leaflet@v4

I've loaded this package and it has worked fine with react-leaflet's v4.0 release in a project. I think the support constraint can be relaxed a bit.
https://github.com/hugobarragon/react-leaflet-drift-marker/blob/b78011d6ad06bc77595b9b57bd39ae7fe364db27/package.json#L53

Hi, just changing that on package json will not work with react v4, its a major version and custom components may have changed, i will have to look into it, and if all apis are implemented, can you please create a code sandbox with reactv4 and this lib to check what seems to be the problem?

My solution, right or wrong, was to have the installer always use --legacy-peer-deps to get around it via an .npmrc file. You're right that my implementation doesn't use every feature of this library, so I can't say that it's fully compatible.
I'm using it at https://map.transitnownash.org/ (vehicles update every seven seconds) - source if you need to see it working.

Great project, so happy its being used :)
I will try to make a v4 to match react-leaflet

https://github.com/hugobarragon/react-leaflet-drift-marker/pull/6


Various erroneous notes on innerHTML page

It seems there are various mistakes on innerHTML, about the notes (partial support).
For example, concerning IE:

IE 6-9 do support insertAdjacentHTML (though it's broken for tables, which is already noted in the known issues) – Refs: insertAdjacentHTML
IE 9 do support parseFromString, according to MDN.

So, IE 6-8 should not have note 2, but note 3, and IE 9 should have no note.
 I also found issues concerning Firefox:

Firefox supports parseFromString since first version, but insertAdjacentHTML since version 8.

So, all current notes for Firefox should be removed, and a 4th note should be created, for Firefox 1-7: "Partial support refers to lacking support for insertAdjacentHTML on the DOMParser."

Just for the record, the Firefox ticket: Bug 613662 - Implement insertAdjacentHTML.

OK, I have all checked again, and the data seems to be entirely correct, good work!
There is only one remaining thing:

IE 6-9 do support insertAdjacentHTML, though it's broken for tables. Currently, you note IE 6-9 don't support insertAdjacentHTML at all.

The current presence of notes emphasizes there are issues with insertAdjacentHTML (also innerHTML?) in IE < 10, even if they do exist.
So I guess it's fine as it is, instead of making it fully green thus making the issues much less noticeable.


As a user who has many friends I want to view different groups of my friends by their tags so that I can manage my large list of contacts well

Implemented by #97


Scan reveal speed

Allow for custom reveal speeds for scans.

Tested and it won't work

Reason: m_revealSpeed is private thus it doesn't being serialized when instantiate it

Probably need to Patch through CP Setup methods postfix and
__instance.m_spline.Cast<CP_Holopath_Spline>().m_revealSpeed = newValue

Tested, Looks good for me

have you tested adding reveal speed to both clusters and bioscans? Both are supported.
e.g. have a cluster than takes 10 seconds to reveal, but have a custom bioscan that takes half a second to reveal.
You should see a really long draw time for the initial cluster, then the bioscans appearing really fast

As per DM's

Change RevealDistanceMode.Distance to RevealDistanceMode.ScaleByDistance to make the functionality clear in the name
Warn the user when the config is in an invalid state (Mode set to time & Speed <= 0 / Mode set to ScaleByDistance and Speed < 0)
Add RevealDistanceMode.Instant to justify using an enum for instantly revealing the scans

Since we're adding customization for the holo path, the ability to hide them would be a "nice to have"

Requesting testing of new changes to implement InstantScan. Also stuff was renamed so be aware...

We need in-game testing

Has becky review it.

Instant scan doesn't appear to work at all, fix is just to have it internally treat it as 0.1 constant time.

Other than that everything seems to work


Add a prisma.$reset() function as shortcut to running prisma migrate reset

Problem
prisma migrate reset is used heavily for testing. I want a prisma.$reset() function added to prisma client as super easy shortcut to reset the database.
Suggested solution
I've added this to our Blitz adapter and it's super nice in Jest tests:
beforeEach(async () => {
  await db.$reset()
})

Optionally this command could be disabled/fail if process.env.NODE_ENV === 'production'
Alternatives
None

Is there an update from the Prisma team on the possible addition of this functionality?
I am facing the same problem and I cannot see any way, even if it is a workaround, to clean up the database (preserving the schema) between E2E tests.

As a workaround, you can copy the code we use in Blitz for this: https://github.com/blitz-js/blitz/blob/canary/packages/core/src/prisma-utils.ts
Which you can then use like this:
import {enhancePrisma} from "./enhancePrisma"
import {PrismaClient} from "@prisma/client"

const EnhancedPrisma = enhancePrisma(PrismaClient)

export * from "@prisma/client"
export default new EnhancedPrisma()

In a similar vein, it would also be nice to be able to run migrations and db push programmatically, for testing purposes.

That would be related to https://github.com/prisma/prisma/issues/4703 @saevarb

Example https://gist.github.com/rsaryev/65884be2d8992552e9a603fefabaecb3

As a workaround, you can copy the code we use in Blitz for this: https://github.com/blitz-js/blitz/blob/canary/packages/core/src/prisma-utils.ts
Which you can then use like this:
import {enhancePrisma} from "./enhancePrisma"
import {PrismaClient} from "@prisma/client"

const EnhancedPrisma = enhancePrisma(PrismaClient)

export * from "@prisma/client"
export default new EnhancedPrisma()

Looks like this has moved to https://github.com/blitz-js/blitz/blob/canary/nextjs/packages/next/stdlib/prisma-utils.ts

FYI I'm migrating from sequelize to prisma and at the moment performance using transaction + executeRaw method by  @rsaryev (https://gist.github.com/rsaryev/65884be2d8992552e9a603fefabaecb3) is close to sequelize's and much more faster (10s per reset in my jest setup) than spawning migrate reset cli each time .

@yue4u Agreed this would be a lot faster but it's also lacking functionality. Running a full migrate reset has the advantage of really resetting the db, including reapplying all migrations. Useful if there is seeded data in migration that are required for the application to function, and therefore required for tests to pass. Generally for e2e tests I'd prefer quality and confidence over speed (within reason of course).

Is there any indication that this will be added in the near future? It would be helpful in multiple projects of me.


:memo: TODO: Add GUI support

As a non-programming background user, I wish to have an interactive dashboard that has the following functions:

TNG transaction Table
eWallet balance level over time
Top N Spending Merchant (Frequency / Total Spent) Including RFID Toll payment
Top N Reload Source (Frequency / Total Reload) Including GO+ Interest
Spending pattern over a week

Expectation:

Scrollable Transaction Table (best if I can interact (click/hover) on the transaction record to view the detail)
Line chart for balance level
Either donut chart or bar chart for Top N transactions
Weekly Heatmap

Suggestion:

Using Selenium + Apache Echarts to visualize the dashboard, to the best of my knowledge
or use Streamlit as suggested by @andytan0727
Considering some users would only like to output CSV instead of viewing the dashboard, make a config file to store user settings

~2. Line chart for balance level~
2. Waterfall diagram for net gain / net spend in the eWallet
Since the balance column is unavailable in the CSV output, I decided to drop this section. However to trace the reload cycle of the eWallet, I suggest to use the WaterFall diagram in this Demo:


dockerTools.pullImage: errors from skopeo about --src-tls-verify (difficult to repro)

Describe the bug
On latest unstable, The new version of the skopeo package (1.4.0, introduced in 335f1878009ff226aeea7fca694a31f3df7f35ea) seems to have changed something and now complains about --src-tls-verify as used here.
Steps To Reproduce
Build this docker package definition:
{ pkgs ? import <nixpkgs> { } }:
pkgs.dockerTools.pullImage
{
    imageName = "alpine";
    imageDigest = pkgs.lib.fakeSha256;
    sha256 = pkgs.lib.fakeSha256;
    finalImageName = "alpine";
    finalImageTag = "latest";
}

on latest nixos-unstable (and nixpkgs-unstable) with skopeo 1.4.0, and you get the following error:
asf@ferdl:~/Mess/current> nix-build hello-docker.nix
these 2 derivations will be built:
  /nix/store/lwwy307wq35dadjynyjd756cbfijrl0x-docker-image-alpine-latest.tar.drv
  /nix/store/0lvh6c9ya6v6s3hb6mc18wdj5cr234kx-docker-image-hello-docker.tar.gz.drv
building '/nix/store/lwwy307wq35dadjynyjd756cbfijrl0x-docker-image-alpine-latest.tar.drv'...
FATA[0000] unknown flag: --src-tls-verify
error: builder for '/nix/store/lwwy307wq35dadjynyjd756cbfijrl0x-docker-image-alpine-latest.tar.drv' failed with exit code 1;
       last 1 log lines:
       > FATA[0000] unknown flag: --src-tls-verify
       For full logs, run 'nix log /nix/store/lwwy307wq35dadjynyjd756cbfijrl0x-docker-image-alpine-latest.tar.drv'.
error: 1 dependencies of derivation '/nix/store/0lvh6c9ya6v6s3hb6mc18wdj5cr234kx-docker-image-hello-docker.tar.gz.drv' failed to build

Weirdly enough other definitions with actual SHA256 values (e.g. some definitions emitted from nix-prefetch-docker) do work, whereas others fail with the same message as above.
Expected behavior
Should not emit that skopeo error
Screenshots
n/a
Additional context
n/a
Notify maintainers
@edwtjo, @zowoq
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
 - system: `"x86_64-linux"`
 - host os: `Linux 5.10.52, NixOS, 21.11 (Porcupine)`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.4pre20210707_02dd6bb`
 - nixpkgs: `/nix/store/0j4qxa4l7hf9fy831ryyd678hq6dcl6f-source`

Maintainer information:
# a list of nixpkgs attributes affected by the problem
attribute: [skopeo]
# a list of nixos modules affected by the problem
module:

Update: I think I have found out what reproduces this issue - not having downloaded the image in your store and in your docker. I'd used nix-prefetch-docker to get the SHAs and image tags & whatnot to test with, which warmed the cache and prevented the code path that triggers that error from getting hit.
I think this may just be it, then: Invoking this version of skopeo with the options in dockerTools will raise the error above.

The work-around is also pretty clear: On the system where you are building the image, usenix-shell -p nix-prefetch-docker and fetch all the base images that you're using. This works because nix-prefetch-docker seems to be based on skopeo 1.3 for me for now... It probably be won't soon, though. /:


fix lost updated t9n stranslate

I found forget update T9n translate in PasswordOrService component.

Awesome, I missed that one, thx!


Multi-step signup form

Description of changes
Adds a multi-step information gathering form that collects advanced information on our users.
Previous flow
signup   =>    thanks>  =>    profile
New flow
signup   =>    signup-info   =>    thanks   =>    profile
Miltary vs Civilian flow
Volunteer flow: http://bit.ly/opcvolunteer
Military flow: http://bit.ly/opcmilitary
Information Collected
military status, branch, the level of schooling, interest in scholarships, current work info, work position, company name, programming language interests, discipline interests.
Usage
While this supplements the current 'new-user' signup, we can also direct current users to
/signup-info to collect advanced information on our existing user base.
Potential Issue
Might be blocked by backend issue pending to create the rest of the fields @rickr
Props Necessary
@sethbergman for all the work on this
@matthewzurek for spending time on planning this thing in the beginning
@rickr for answering an a$$load of questions
Issue Resolved

Fixes #265

Fixes #407


Flexible arrays not properly created from zero-length arrays.

Is it possible? For example, structure contains cstring, or some array.

The structure editor allows you to add "flexible arrays."  You do this by specifying an array length of 0.  The is a section in the the Structure Editor's help called "Flexible Array Component" if you want to read more about it.

It is not true.

Also:

So, it only works with that button, but not with the manual input:
https://youtu.be/JNeiBS82CY8

It must be marked as a bug!

I'll reopen it as a bug for now and consult the designer of the feature.  Thanks!

@ghidra1, can you give you opinion?

I got a bit more info on this for you.  Ghidra does not support 0-length DataType's, which is why we can't parse byte[0] in that table field in the structure editor.  The help correctly describes the only way to add flexible arrays.

What about pointer to variable size table ?

You would simply add a pointer component


Improve state transitions of codec::* structs

Current
The current state transitions in the codec module are as follows:
1 context
2 └── opened
3     ├── decoder
      │   ├── audio
      │   ├── subtitles
      │   └── video
4     └── encoder
          ├── audio
          ├── subtitles
          └── video

A Context is obtained from a Stream object (or created anew).
This context is opened using the right Codec with avcodec_open2().
A possible Decoder is created from the opened Context.
A possible Encoder is created from the opened Context.

Fields from AVCodecContext are exposed with getter/setter methods (primarily) from step 3/4 onwards.
This scheme works great for decoding as we are intrested in the fields in AVCodecContext set by the demuxer (avformat_find_stream_info()) or the encoder (avcodec_open2()).
However this does not work for encoding. libavcodec seems to expect certain fields in AVCodecContext set before calling avcodec_open2() (for example the time_base field, see here).  The current architecture does not allow this, as the setters are defined below the Encoder state (and avcodec_open2() needs to be called in step 2.).
New
I propose the following new schema:
1   context
2   ├── decoder_context
2.1 │   └── opened_context
2.2 │       ├── audio
2.3 │       ├── subtitles
2.4 │       └── video
3   └── encoder_context
3.1     ├── audio
        │   └── opened
3.2     ├── subtitles
        │   └── opened
3.3     └── video
           └── opened

A Context is obtained from a Stream object (or created anew).
Common setters (like set_threading()) are defined here.
A 'Decoder' can be created by state transition. This decoder exposes getter methods for fields set by the demuxer (like bits_per_coded_sample see here).

The Decoder can be opened (avcodec_open2() in the background plus state transition). This opened decoder exposes more getter functions for fields set by the decoder common to all streams.
An opened Audio decoder exposes getter functions for audio-specific fields and a decode() function.
An opened Subtitles decoder exposes getter functions for subtitle-specific fields and a decode() function.
An opened Video decoder exposes getter functions for video-specific fields and a decode() function.

An Encoder can be created by state transition.

An Audio encoder can be created by state transition which exposes audio encoder specific setter functions. This can be opened with avcodec_open2() and a state transition to expose a encode() function.
An Subtitles encoder can be created by state transition which exposes subtitle encoder specific setter functions. This can be opened with avcodec_open2() and a state transition to expose a encode() function.
An Video encoder can be created by state transition which exposes video encoder specific setter functions. This can be opened with avcodec_open2() and a state transition to expose a encode() function.

I would start on implementing this to see if its possible and ergonomic.  Feedback is welcome :).

Sounds good, do you have any code examples so it's a little more clear from the API side how things will look?

Also to avoid trumping on your work, I'm going to do the Stream and StreamMut changes now.

https://github.com/lummax/rust-ffmpeg/tree/improve-codec-transitions contains the first part (decoder) of the changes. See https://github.com/lummax/rust-ffmpeg/blob/improve-codec-transitions/examples/metadata.rs#L25 for the changed API.

Sounds good, just please leave time_base in even if it's deprecated, I use it :stuck_out_tongue:

Do you use in on AVCodeContext or on AVStream? Because only the use on AVCodeContext (if decoding) is deprecated.

Ah, I use it on AVStream, feel free to remove it altogether then.


Click event firing twice

When opening DatePicker , it will select date automatically.
$event.stopPropagation() is also not solving the issue.
Please reply ASAP.

Please check the demo app. I tried to re produce this issue in iOS and android, but it's not firing twice and not auto selecting the date. Please cross check your code also.


Cannot narrow down "[0, null] | [number, T]" with type guard

TypeScript Version:  3.9.6

Search Terms: sum types
Code
class T {
	asdf: string = ""
}

function works(a: [0, null] | [1, T]) {
	if (a[0] !== 0) {
		let x = a[1]
		console.log(x.asdf)
	}
}

function doesnt_work(a: [0, null] | [number, T]) {
	if (a[0] !== 0) {
		let x = a[1]
		console.log(x.asdf)
	}
}

Expected behavior:
Should compile. This is no way to pass [x, null] to doesnt_work if x !== 0, so if a[0] !== 0, then a[1] !== null
Actual behavior:
Fails to compile. Although the simpler case works
Playground Link: https://www.typescriptlang.org/play/?ts=4.0.0-dev.20200708#code/MYGwhgzhAEAq0G8BQBISATAZgLmhALgE4CWAdgObQC80ARLUgL5JKYCupw+xA9qdAHcehANYQAFGFwBtAAwAaaKTYgQAXWgAfaNICMi2GoCUiVMUzRJcjQEIqNWSeQoUIAKb5oAD2rQwetVQUYD4IHncAOhAecnEvCIxMI1RmZlYOLl5+dB43CFJ8AH0hUUkZBSUVdS0dZQBbACM3QgNjUxRzS39ZW3toR3bXD29ff11AlxDSMMjo2PjE5JRUpCA
Related Issues:

a: [number, T] includes values like [0, new T()], so there's no way to narrow it out with a[0] ===0.
Note that you can check for null.
class T {
	asdf: string = ""
}

function works(a: [0, null] | [1, T]) {
	if (a[0] !== 0) {
		let x = a[1]
		console.log(x.asdf)
	}
}

function doesnt_work(a: [0, null] | [number, T]) {
	if (a[1] !== null) {
		let x = a[1]
		console.log(x.asdf)
	}
}

a: [number, T] includes values like [0, new T()]

You are right.
However, this only means you cannot go from a[1] !== null to a[0] !== 0. OTOH you can do from a[0] !== 0 to a[1] !== null.


Ability to add new columns before using csvstack to merge file with extra columns

Sometimes I want to merge two files, but one file has an additional column that I want to preserve. Let's call these two files: three_columns.csv and four_columns.csv. In order to use csvstack to merge them, I first want to make sure they'll have the same columns, so I first create an empty new column on three_columns.csv:
awk '{ print $0 "," }' < three_columns.csv
If the order of the columns are different, I might then need to process one with csvcut and then I can merge them with csvstack
csvstack three_columns.csv four_columns.csv > merged_file.csv
However three_columns.csv doesn't have the heading of the new column, so I need to manually replace the column headings for the whole file, eg:
var="These,Are,My,Columns"
sed "1s/.*/$var/" merged_file.csv

Is there a better way of doing this?
It might be nice if there was a simpler way to add new columns and rename column headings using a command packaged with csvkit.

Just experienced this exact same situation and, after reading @onyxfish 's argument, I wanted to say that, IMHO, this is not editing at all, but a very common use case in which you want to stack 2 or more csv files with only some corresponding columns. It's analogous to the csvjoin command, but transposed to rows instead of columns and, as such, it could be solved similarly as it was it was solved there with --outer, --left and --right.
There should (or at least I'd be very happy if there was) be, at least, a modifier (eg. --all) which tells the csvstack command to keep all the involved files' columns for the output file, and just leave the values null for rows coming from files without that column.

I think this may be solved when #245 is closed.


Login requiring Pages

Architecture thoughts ...

Preroute - check if login is required
Load login-form instead - with originally requested page attached as url parameter
After login, replace login page with originally required page


Over-determined migration defaults

#47 added minimal support for migration defaults.  We are missing tests of some invalid default cases, where fields from Symmetric and Asymmetric migrations are all jumbled together to create an ambiguity.

We already have tests for this


gnomeExtensions.clipboard-indicator: 34 -> 37

Motivation for this change
clipboard-indicator has a new release. Most notably, this adds support
for GNOME 3.38 and drops support for 3.36 (note that while the extension
no longer lists 3.36 as a supported version, it does still work).
Full changelog
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

@NixOS/gnome Now that 3.38 is on master, can I get a review on this update? The extension is broken on 3.38 without this version bump.

Changes look fine to me.


linkerSettings appears to work differently on macOS than linux

Description
I was just trying to pass -no-toolchain-stdlib-rpath to the build when installing sourcekit-lsp in apple/sourcekit-lsp#715 and Alex ran a full toolchain build to make sure it worked well. Strangely, it passed on linux and Windows builds sourcekit-lsp using CMake, not SPM, so it only failed on macOS, because it appears to incorrectly pass this Swift linker flag to clang:
0%: Link sourcekit-lsp (x86_64)
clang: error: unknown argument: '-no-toolchain-stdlib-rpath'

I'm not sure why it does this on macOS only, as the SPM source clearly shows the linker command-line running through the Swift compiler.
Expected behavior
No differences in linkerSettings behavior on different platforms
Actual behavior
The given example works on linux, but not macOS.
Steps to reproduce
Add a linkerSettings with a Swift flag like -no-toolchain-stdlib-rpath and build on macOS.
Swift Package Manager version/commit hash
1750a26743a672c62e201e7d2477e81d26a466b2
Swift & OS version (output of swift --version && uname -a)
Taken from the linked macOS CI run as I don't build on macOS:
Swift compiler commit - 642386290e84e9eeeed8ba70f07d5c3628be728a

That build seems to be using --build-system xcode which uses clang as the linker driver, not Swift.

Alright, should we simply document this somewhere or is a change needed? It is convenient to be able to pass Swift compiler flags like this that affect linking, but it can't be done for this one build config of XCode.

I wonder if we ultimately need a way to differentiate? For now, I'd suggest just checking #if os(macOS) to not pass it if that is true.

I wonder if we ultimately need a way to differentiate?

Unsure what you mean, have a way to pass different flags if it's the XCode build config of SPM?

For now, I'd suggest just checking #if os(macOS) to not pass it if that is true.

Oh, went one better and only pass it for the linux target now, that pull was merged.

Unsure what you mean, have a way to pass different flags if it's the XCode build config of SPM?

I think it would be on the axis of whether the linker driver is clang or swiftc, maybe something like unsafeSwiftFlags/unsafeCFlags in addition to unsafeFlags.

Interesting, that might work.


In json reader, should the span array be terminated with AZ_SPAN_NULL instead of explicit length parameter?

From https://github.com/Azure/azure-sdk-for-c/pull/1023#discussion_r468060598
Consider removing the number_of_buffers parameter and changing:
AZ_NODISCARD az_result az_json_reader_chunked_init(
    az_json_reader* json_reader,
    az_span json_buffers[],
    int32_t number_of_buffers,
    az_json_reader_options const* options);

To:
AZ_NODISCARD az_result az_json_reader_chunked_init(
    az_json_reader* json_reader,
    az_span json_buffers[],
    az_json_reader_options const* options);

@ahsonkhan @JeffreyRichter If we are going to change this api we need to do it before GA.


works fine in case person is standing little longer distance but not when person is at shorter distance

Sometimes it do not detect the person properly.. suddenly when we raise hand in the video. Video is very much distorted. How to make this functionality work more accurate ?

You can try using a different model. Bodypix is not always optimal but was the only suitable one at time of making this. Since then many other models in TensorFlow.js for human body detection.


publish.include has a lower priority than .gitignore

The directories or files listed in the .gitignore are missing even if they are explicitly declared in publish.include.
According to the document:

jsr publish will ignore files that are listed in a .gitignore file in the root of your package. Additionally, you can specify the exclude and include fields in your jsr.json / deno.json file to ignore or include specific files.

@cloydlau do you have an example that reproduces this behaviour.

clone https://github.com/cloydlau/json-editor-vue.git
pnpm i
pnpm build
npx jsr publish --dry-run --allow-dirty
dist/** is listed in publish.include, but missing from the files that are about to be published

I think this is intentional, and you probably have to add an "exclude": [ "!dist/**" ] (ie do not exclude dist/).
@dsherret can you confirm?
If that's right, this just needs a docs update.

"include": ["dist"] works, thanks.


Room summary

Make the recents screen use MXRoomSummary introduced by https://github.com/matrix-org/matrix-ios-sdk/pull/232.
MXKRecentsDataSource does not depend anymore on MXKRoomDataSource.

Hi manuroe,
As suggested i have made changes to MXKRecentsDataSource
Made the recents screen use MXRoomSummary introduced by
matrix-org/matrix-ios-sdk#232 [1].
But am facing the following crash
-[MXKEventFormatter session:updateRoomSummary:withLastEvent:state:]: unrecognized selector sent to instance 0x1703a2f40

@swapnapentam: Sorry, I missed your comment. I fixed it yesterday.


CARLA Hangs during Execution: Seeking Insights on GDB Thread Analysis

CARLA version: 0.9.10
Platform/OS: 18.04.1-Ubuntu
Problem you have experienced: CARLA hangs during execution
What you expected to happen: maybe there's a deadlock
Steps to reproduce: maybe it is triggered under certain conditions, so I've attached a full stack analysis
Other information (documentation you consulted, workarounds you tried):
I am encountering an issue where my python(pytorch) program, which interacts with the CARLA simulator, hangs during execution. To investigate this, I have attached a gdb file with thread analysis (gdb.txt) captured at the point where the program becomes unresponsive.
Issue Description:

The program involves multiple threads and seems to be stuck in a waiting state.
Several threads are in a wait state, particularly with calls to futex_wait and PyTorch's autograd engine functions.
One thread appears to be waiting on a CUDA-related operation (libcuda.so), suggesting a potential wait for GPU resources or a GPU operation to complete.
The main thread is waiting in a CARLA function, specifically in carla::traffic_manager::TrafficManagerLocal::SynchronousTick.

Problem figures for references:

I am attaching the gdb.txt for reference and would greatly appreciate any insights or suggestions on how to diagnose and resolve this issue. The gdb output includes the state of all threads at the time of the hang.
Thank you for your assistance!

I just looked for similar issues elsewhere. I found it at below site.
https://github.com/microsoft/onnxruntime/issues/13993
That issue seemed to be solved by changing LD_LIBRARY_PATH.
Kindly check it.

@SasukeBLX LD_LIBRARY_PATH is a broad concept and the issues mentioned in the provided link do not seem to closely resemble the situation I'm encountering with CARLA. Therefore, I'm unable to determine a further solution at this point. Thank you for your attention and I welcome any additional suggestions.

I get a new thread analysis file gdb.txt when I encounter the question again. I think there are some common issues that may be helpful to solve the problem. It seems the main thread using the Carla server is stuck around the following functions:

carla::traffic_manager::TrafficManager::Tick
carla::client::detail::Simulator::Tick
carla::client::World::Tick

I hope this file will be helpful in addressing these issues.


TypeError: interceptors.map is not a function

I used this code and it threw an error
const { RequestInterceptor } = require('node-request-interceptor')
const withDefaultInterceptors = require('node-request-interceptor/lib/presets/default')

const interceptor = new RequestInterceptor(withDefaultInterceptors)

Node says that const interceptor = new RequestInterceptor(withDefaultInterceptors) threw the error

Hey, @vismodo. Not sure what you're trying to do but that example doesn't look correct: there's no RequestInterceptor export from this library.
If you want to combine multiple interceptors, you must use BatchInterceptor as described in the docs:
https://github.com/mswjs/interceptors#presets

We export the Interceptor class but it's an abstract class that every interceptor needs to implement. You likely don't need it at all for your use case unless you're building a custom interceptor (which you, probably, shouldn't do in the first place).

@vismodo that's the old version of the package. Please, always check the package version vs the latest published version on GitHub. That package has been discontinued for years (no such thing as "unpubilsh a package" in NPM).


Missed updates for a large Lerna repo

Hello,
I am surprised @dependabot did not submit PR's for a number of package.json files at are monitored in our repository, while there are newer versions available for certain of our direct dependencies, as can be seen in aws/aws-cdk#3545.
I want to count on @dependabot for not having to manually cater for dependency upgrades, so I'd like to get an understanding of why I didn't get automated PRs for those dependencies (even after manually clicking "bump now").
Thank,
Romain.

👋 This is a pretty old ticket and we've shipped a lot of changes / improvements since then.
Is this still happening or should this issue be closed?


Synchronous FIFO submission

Thanks again for the help :)

You're welcome!
Can you please also add the documentation into the info.yaml in this repo (and add it to the pull request)?

@urish added!


Possible to do a split (and loop) on an output variable?

We are creating a comma separated list of files as an output variable during one of our stages:
- script: |
          set -ex
          cd ./packerlogs
          SUCCESSFUL_PACKERLOGS=$(echo $(grep -l "^ManagedImageSharedImageGalleryId:" ./packerlog-*cdh* | xargs -n1 basename) | tr ' ' ,)
          echo "##vso[task.setvariable variable=PackerlogList;isOutput=true]$SUCCESSFUL_PACKERLOGS"
        condition: and(succeeded(), ne(variables['CheckForExistingPackerlogs.ExistingPackerlogs'], 'false'))
        name: SetPackerlogList

The value of the output variable will be something like "packerlog-cdh-xxx,packerlog-cdh-yyy,...". The amount of comma separated packerlogs is dynamic, not static.
We then want to loop over this output variable in the next stage using the new split expression:
- job: Terraform
    dependsOn: GetSubscriptionId
    displayName: terraform
    variables:
      PackerlogList: $[ stageDependencies.AnsibleGoldenImageBuild.AnsibleGoldenImageBuild.outputs['SetPackerlogList.PackerlogList'] ]
      TargetSubscriptionId: $[ dependencies.GetSubscriptionId.outputs['SetTargetSubscriptionId.TargetSubscriptionId'] ]

    pool:
      name: Agent Pool Linux VMSS

    steps:
      - ${{ each file in split(variables.PackerlogList, ',')}}:
        - script: |
            set -ex
            echo "List: $(PackerlogList)"
            echo "File: ${{ file }}"
          displayName: Display PackerlogList and current file value

However, this unfortunately fails with the following error:
2022-10-28T13:33:12.5568993Z + echo 'List: packerlog-cdh-xxx,packerlog-cdh-yyy'
2022-10-28T13:33:12.5570653Z /agent/_work/_temp/f9946e18-76ff-4001-9e99-b1dab8350344.sh: line 3: stageDependencies.AnsibleGoldenImageBuild.AnsibleGoldenImageBuild.outputs['SetPackerLogList.PackerlogList'] : syntax error: invalid arithmetic operator (error token is ".AnsibleGoldenImageBuild.AnsibleGoldenImageBuild.outputs['SetPackerLogList.PackerlogList'] ")
2022-10-28T13:33:12.5571897Z List: packerlog-cdh-xxx,packerlog-cdh-yyy
2022-10-28T13:33:12.5596137Z ##[error]Bash exited with code '1'.
2022-10-28T13:33:12.5629775Z ##[section]Finishing: Display PackerLogList and current file value

So my question is basically: Is something like this even possible? If yes, what would the syntax be like?
Cheers,
Robin

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 77c58a78-a567-e99a-9eb7-62dddd1b90b6
Version Independent ID: 680a79bc-11de-39fc-43e3-e07dc762db18
Content: Expressions - Azure Pipelines
Content Source: docs/pipelines/process/expressions.md
Service: azure-devops-pipelines
Sub-service: azure-devops-pipelines-process
GitHub Login: @juliakm
Microsoft Alias: jukullam

@RobinZergiebel -- Robin, thank you for your question.  Please consider these resources:

Azure DevOps Services
Azure DevOps on Stack Overflow

@juliakm -- Julia, please look into this issue.

I have a similar requirement to loop on output variables.
@WilliamAntonRohm  @juliakm can you please confirm if this is possible or not?

For everyone running into the same problem (e.g. @pulkitpatel95):
As stated above, I doubt that this is possible due to the split and loop being rendered at compile time.
We have found a different (though maybe less elegant) solution:

Pipeline 1 writes a vars.yml file and commits it to Git
Pipeline 2 triggers on the commit and runs the split and loop on the variable from the vars.yml

Example code:

Pipeline 1:

stages:
- stage: Commit
  displayName: Commit vars.yml

  jobs:
  - job: Commit
    displayName: Commit vars.yml
    pool:
      vmImage: 'ubuntu-latest'

    steps:
      - checkout: self
        persistCredentials: true

      - script: |
          git config --global user.email<EMAIL_ADDRESS>& git config --global user.name "Robin Test"
        displayName: Configure git config
        workingDirectory: $(System.DefaultWorkingDirectory)

      - script: |
          git checkout -b auto/$(Build.BuildId)
          find $(System.DefaultWorkingDirectory) -name "vars.yml" -type f -exec sed -i -E "s/PackerlogList: .*/PackerlogList: $(TestList)/g" {} \;
          git add -A
          git commit -am "Pipeline $(Build.BuildId)"
          git push --set-upstream origin auto/$(Build.BuildId)
        displayName: Edit vars.yml and git push
        workingDirectory: $(System.DefaultWorkingDirectory)

Pipeline 2:

trigger:
  branches:
    include:
    - auto/*

stages:
- stage: Split
  displayName: Test split

  jobs:
  - job: Split
    displayName: Test split
    variables:
    - template: vars.yml
    pool:
      vmImage: 'ubuntu-latest'

    steps:
      - ${{ each file in split(variables.PackerlogList, ',')}}:
        - script: |
            echo "File: ${{ file }}"
          displayName: Display current file value

vars.yml:

# File: vars.yml
variables:
  PackerlogList: packerlog-cdh-aaa,packerlog-cdh-bbb

Having tested various ways to attempt this and failing, based on my observations, the previous statement is mostly true. What I did notice is that if I referred to my variable like this:

${{each t in split('$(functionList)', ',')}}:

script: |
echo '${{t}}'

It does not fail, and it even produces a single line output that includes my comma separated values. Of course I expected it to break that into an array for processing which it failed to do.

I have the same problem:
The following code
  variables:
    build: $(Build.BuildNumber)
  steps:
  - script: |
      echo Build: $(build)
      echo split variable reference using variables.build: ${{ split(variables.build, '.')[0] }}
      echo split variable reference using variables bracket notation ${{ split(variables['build'], '.')[0] }}

produces the following output:
Build: SDFramework_NSBDemo_master_2.0.24096.51
Version:
split variable reference using variables.build: $(Build
split variable reference using variables bracket notation $(Build

The last 2 lines show that the split function actually splits the variable name, instead of the variable value.