Multiroom support

Hi, do you think multiroom/groups support be possible to play simultaneously on multiple alexa devices?
Thank you.

Hi @robyarrow1,
I don't think this will be possible, I've attempted similar things in the past and keeping the devices perfectly in sync is extremely difficult.  I find that when you get a device even a second or two out of sync the sound become quite annoying.  There are other projects that attempt to do this with Logitech Media Server like https://www.max2play.com that might be better suited to your use case.

I asked chatgpt to see if this could be done as I know very little coding, but like to play around with things and this is what it came up with, I tried to add the code myself to it but couldn't get it work, I wonder if it's something you could use? I'll post the info here, but it basically came up with 2 options.
Option 1
To enable AskNavidrome to work on groups of Alexa-enabled speakers, we’ll need to modify a few parts of the code. Here’s how you can approach it step by step:

Modify the Interaction Model to Support Groups
First, in the interactionModel file (usually model.json for Alexa skills), add intents that support group commands.

json
{
  "intents": [
    {
      "name": "PlayInGroupIntent",
      "slots": [
        {
          "name": "GroupName",
          "type": "AMAZON.Room"
        }
      ],
      "samples": [
        "play music in {GroupName}",
        "start playing in {GroupName}",
        "play in {GroupName}"
      ]
    }
  ]
}

This will allow the skill to recognize commands like "Play in the Living Room" or "Play on all speakers."

Update the Skill Logic to Support Speaker Groups
In the skill's backend (probably using a Lambda function in Node.js or Python), you'll need to handle group playback requests by passing a group name to the playback logic.

If you're using Node.js, the core Alexa SDK would look something like this:
javascript
const Alexa = require('ask-sdk-core');

// Function to play on a group of speakers
const PlayInGroupIntentHandler = {
  canHandle(handlerInput) {
    return Alexa.getRequestType(handlerInput.requestEnvelope) === 'IntentRequest'
      && Alexa.getIntentName(handlerInput.requestEnvelope) === 'PlayInGroupIntent';
  },
  handle(handlerInput) {
    const groupName = Alexa.getSlotValue(handlerInput.requestEnvelope, 'GroupName');

    // Construct the response that would trigger playback in the speaker group
    const audioUrl = 'https://your-subsonic-server.com/stream-url'; // Your Navidrome stream URL

    return handlerInput.responseBuilder
      .speak(`Playing in ${groupName}`)
      .addAudioPlayerPlayDirective('REPLACE_ALL', audioUrl, audioUrl, 0, null)
      .getResponse();
  }
};

Here, the addAudioPlayerPlayDirective method triggers audio playback, but we need to associate the stream with a group. Unfortunately, the current Alexa API doesn't provide direct multi-room control in the SDK, but by invoking Alexa’s built-in group functionality, you can address multi-room groups by name.

Target Group Playback in Subsonic-Navidrome Backend
Ensure your Subsonic server is capable of handling multiple requests and can stream to multiple endpoints (in this case, Alexa devices). The modification needs to provide the stream URL, which Alexa will handle.

To integrate group selection, you’ll need to adjust the stream logic to work with multiple endpoints, especially if using Navidrome or similar API-supported media servers.
For instance, if you're using a Python-based backend:
python
def play_in_group(group_name, stream_url):
    # Logic to initiate stream in group
    return {
        'response': f"Playing music in {group_name}",
        'stream': stream_url  # Use Navidrome/Subsonic stream
    }

Test the Alexa Skill and Groups
Ensure you've set up the group of Alexa devices through the Alexa app. You can name the group (e.g., "Living Room"). When you test the skill with commands like, "Alexa, play in the living room," it should recognize the group and direct the stream to that set of speakers.

Handle Device or Group Context in Requests
To get the group context or device ID, you can use the following method:

javascript
const requestEnvelope = handlerInput.requestEnvelope;
const deviceId = requestEnvelope.context.System.device.deviceId;

// Optionally check group or linked devices using Alexa’s multi-room support
Notes:
Alexa SDK does not natively support managing speaker groups, but invoking Alexa's own multi-room music setup via voice commands should handle this.
You might need to ensure that all devices are within the same Alexa multi-room setup to enable synchronized playback.
Next Steps:
Adjust the Alexa Skill manifest to request appropriate permissions (like alexa::audio::content).
Extend the Navidrome integration to manage multiple connections.
This setup should allow you to add multi-room playback support using existing Alexa functionality, alongside Navidrome's capabilities(
LibHunt
)(
LibHunt
).
Option 2
Overview of Steps
Understand Alexa Multi-Room Music (MRM) Capabilities
Modify the Skill's Interaction Model
Handle Device Groups in the Backend
Update the Skill's Logic to Target Groups
Testing and Validation

Understand Alexa Multi-Room Music (MRM) Capabilities
Alexa's Multi-Room Music feature allows users to play music simultaneously on multiple Echo devices grouped together. While Amazon provides built-in support for MRM, integrating this with a custom skill like AskNavidrome requires leveraging Alexa's APIs to control playback across these groups.

Key Concepts:
Speaker Groups: Defined groups of Echo devices (e.g., "Living Room," "Everywhere").
Device IDs: Unique identifiers for each Echo device.
Playback Control: Commands sent to Alexa to manage music playback.
2. Modify the Skill's Interaction Model
To allow users to specify groups when issuing voice commands, you'll need to update the skill's interaction model to recognize group names.
a. Update Intents
Add new intents or update existing ones to handle group-based commands.
Example: Adding a PlayOnGroupIntent
json
{
  "interactionModel": {
    "languageModel": {
      "intents": [
        {
          "name": "PlayOnGroupIntent",
          "slots": [
            {
              "name": "GroupName",
              "type": "GROUP_NAME"
            },
            {
              "name": "MusicName",
              "type": "AMAZON.Music"
            }
          ],
          "samples": [
            "play {MusicName} on {GroupName}",
            "start {MusicName} in {GroupName}",
            "play music on {GroupName}"
          ]
        },
        // ... other intents
      ],
      "types": [
        {
          "name": "GROUP_NAME",
          "values": [
            { "name": { "value": "Living Room" } },
            { "name": { "value": "Kitchen" } },
            { "name": { "value": "Everywhere" } }
            // Add more group names as per user setup
          ]
        }
      ]
    }
  }
}

b. Define Slot Types
Create a custom slot type GROUP_NAME to recognize user-defined speaker groups.

Handle Device Groups in the Backend
The backend needs to map group names to device IDs or utilize Alexa's built-in groupings.

a. Retrieve Device Groups
Unfortunately, Alexa Skills do not have direct access to manage or retrieve speaker groups. However, you can leverage the Alexa APIs to interact with groups if you have the necessary permissions.
Note: Accessing device groups may require implementing Account Linking and using the Alexa Voice Services (AVS) APIs, which can be complex and might need AWS resources.
For simplicity, we'll assume that users will specify the group name, and the backend will map this to a predefined list of device IDs.
b. Store Group Mappings
Maintain a configuration file or database that maps group names to device IDs.
Example: groups.json
json

{
  "Living Room": ["amzn1.echo-api.device.[DEVICE_ID_1]"],
  "Kitchen": ["amzn1.echo-api.device.[DEVICE_ID_2]"],
  "Everywhere": [
    "amzn1.echo-api.device.[DEVICE_ID_1]",
    "amzn1.echo-api.device.[DEVICE_ID_2]",
    "amzn1.echo-api.device.[DEVICE_ID_3]"
  ]
}

Ensure to secure and manage device IDs appropriately.

Update the Skill's Logic to Target Groups
Modify the backend code to handle group playback based on user input.

a. Sample Code Modification
Assuming your backend is written in Python using Flask-Ask or a similar framework.
Original Playback Function (Single Device):
python
def play_music_on_device(device_id, music_name):
    # Logic to stream music to a single device using Subsonic API
    # Implementation details...
    pass

Modified Playback Function (Group Devices):
python
import json

# Load group mappings
with open('groups.json') as f:
    GROUPS = json.load(f)

def play_music_on_group(group_name, music_name):
    device_ids = GROUPS.get(group_name)
    if not device_ids:
        # Handle unknown group
        return "I couldn't find the group you specified."

    for device_id in device_ids:
        play_music_on_device(device_id, music_name)

    return f"Playing {music_name} on {group_name}."

b. Update Intent Handler
Handle the new PlayOnGroupIntent in your Flask-Ask route.
python
from flask import Flask
from flask_ask import Ask, statement, question, session
import json

app = Flask(__name__)
ask = Ask(app, '/')

# Load group mappings
with open('groups.json') as f:
    GROUPS = json.load(f)

@ask.intent("PlayOnGroupIntent", mapping={'GroupName': 'GroupName', 'MusicName': 'MusicName'})
def play_on_group(GroupName, MusicName):
    if GroupName is None:
        return question("Which group would you like to play music on?")

    group_name = GroupName.title()
    music_name = MusicName if MusicName else "your music"

    device_ids = GROUPS.get(group_name)
    if not device_ids:
        return statement(f"I couldn't find the group named {group_name}.")

    # Implement the play_music_on_group function
    play_music_on_group(group_name, music_name)

    return statement(f"Playing {music_name} on {group_name}.")

def play_music_on_group(group_name, music_name):
    device_ids = GROUPS.get(group_name)
    for device_id in device_ids:
        play_music_on_device(device_id, music_name)

Explanation:
Intent Handler: The play_on_group function handles the PlayOnGroupIntent, retrieves the group name and music name from user input, and calls the play_music_on_group function.
play_music_on_group: Iterates through all device IDs in the specified group and invokes play_music_on_device for each.
c. Implement play_music_on_device
This function interacts with Navidrome's API to stream music to individual devices.
Example Implementation:
python

import requests

NAVIDROME_URL = "https://your-navidrome-server.test"
NAVIDROME_USER = "your_username"
NAVIDROME_PASS = "your_password"

def play_music_on_device(device_id, music_name):
    # Find the track URL from Navidrome
    search_url = f"{NAVIDROME_URL}/rest/search"
    params = {
        'query': music_name,
        'u': NAVIDROME_USER,
        'p': NAVIDROME_PASS,
        'f': 'json'
    }
    response = requests.get(search_url, params=params)
    data = response.json()

    # Extract the first matching song
    try:
        song = data['searchResult2']['song'][0]
        stream_url = song['streamUrl']
    except (KeyError, IndexError):
        print(f"No results found for {music_name}")
        return

    # Send the stream URL to the Alexa device using the AudioPlayer interface
    # This requires implementing the AudioPlayer directives
    # Example:
    # response = {
    #     'version': '1.0',
    #     'response': {
    #         'directives': [
    #             {
    #                 'type': 'AudioPlayer.Play',
    #                 'playBehavior': 'REPLACE_ALL',
    #                 'audioItem': {
    #                     'stream': {
    #                         'token': 'some_token',
    #                         'url': stream_url,
    #                         'offsetInMilliseconds': 0
    #                     }
    #                 }
    #             }
    #         ],
    #         'shouldEndSession': True
    #     }
    # }
    # return response

    # **Note:** Implementing AudioPlayer requires proper handling of directives and response structure.
    # Depending on your existing setup, you might need to adjust accordingly.
    pass

Important Considerations:
Authentication: Ensure secure handling of Navidrome credentials.
Error Handling: Implement robust error handling for API responses.
AudioPlayer Interface: Properly manage AudioPlayer directives to control playback on Alexa devices.

Testing and Validation
After implementing the above changes, thoroughly test the skill to ensure:

Group Recognition: The skill correctly identifies and maps group names to device IDs.
Simultaneous Playback: Music plays concurrently across all devices in the specified group.
Error Handling: The skill gracefully handles scenarios where groups or music tracks are not found.
Voice Command Accuracy: Alexa accurately interprets user commands involving groups.
Testing Steps:
Set Up Speaker Groups:
In the Alexa app, create speaker groups (e.g., "Living Room," "Kitchen").
Note the group names for use in voice commands.
Invoke the Skill:
Use voice commands like:
"Alexa, ask AskNavidrome to play jazz on Living Room."
"Alexa, ask AskNavidrome to start my playlist in Kitchen."
Verify Playback:
Ensure that the specified music plays on all devices within the targeted group simultaneously.
Monitor Logs:
Check backend logs for any errors or unexpected behavior during testing.
Additional Enhancements
To further enhance the multi-room functionality, consider the following:
Dynamic Group Retrieval:
Integrate with Alexa's APIs to dynamically fetch available speaker groups instead of using a static groups.json file.
Note: This requires implementing Account Linking and handling OAuth flows.
User-Specific Configurations:
Allow users to configure their speaker groups within the skill, storing mappings per user.
Advanced Playback Controls:
Implement additional controls like pause, resume, stop for group playback.
Handle synchronization to ensure seamless playback across devices.
Security Enhancements:
Securely manage device IDs and user data.
Implement rate limiting and input validation to prevent misuse.
Conclusion
Integrating multi-room playback into the AskNavidrome Alexa skill involves updating both the interaction model and backend logic to handle speaker groups. The provided code examples offer a foundational approach to implementing this feature. Depending on your specific setup and requirements, you may need to adjust and expand upon these examples.

Hi @robyarrow1
Thanks for this Rob, I'll have a look.
Best regards.


get_variables_in_module() failed to get local variables in module

Dear deepminder:
In some scenario I intend to wrap the tf.metrics into a sonnet module for convenience, the following the wrapper class:
class Metrics(snt.AbstractModule):
    def __init__(self, indicator, name = "metrics"):
        super(Metrics, self).__init__(name = name)
        self._indicator = indicator

    def _build(self, labels, logits):

        if self._indicator == "accuracy":
            metric, metric_update = tf.metrics.accuracy(labels, logits)
            with tf.control_dependencies([metric_update]):
                outputs = tf.identity(metric)
        elif self._indicator == "precision":
            metric, metric_update = tf.metrics.precision(labels, logits)
            with tf.control_dependencies([metric_update]):
                outputs = tf.identity(metric)
        elif self._indicator == "recall":
            metric, metric_update = tf.metrics.recall(labels, logits)
            with tf.control_dependencies([metric_update]):
                outputs = tf.identity(metric)
        elif self._indicator == "f1_score":
            metric_recall, metric_update_recall = tf.metrics.recall(labels, logits)
            metric_precision, metric_update_precision = tf.metrics.precision(labels, logits)
            with tf.control_dependencies([metric_update_recall, metric_update_precision]):
                outputs = 2.0 / (1.0 / metric_recall + 1.0 / metric_precision)
        else:
            raise ValueError("unsupported metrics")

        return outputs

The test code is as follows:
def test2():
    import numpy as np

    labels = tf.constant([1, 0, 0, 0, 0, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0], tf.int32)
    logits = tf.constant([1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0], tf.int32)

    metrics = Metrics("accuracy")
    accuracy = metrics(labels, logits)

    with tf.Session() as sess:
        sess.run([tf.global_variables_initializer(), tf.local_variables_initializer()]) # NOT applicable in practice

        # variables_names = [v.name for v in snt.get_variables_in_module(metrics)] # Nothing returns
        variables_names = [v.name for v in tf.get_collection(tf.GraphKeys.LOCAL_VARIABLES, scope="metrics")] # current work-around

        values = sess.run(variables_names)
        for k, v in zip(variables_names, values):
            print "Variable: ", k
            print "Shape: ", v.shape
            print v

        '''
        score = sess.run(accuracy)
        print(score)
        '''

if __name__ == "__main__":
    test2()

If I want to inspect the metrics for each batch, I cannot rely tf.local_variables_initializer() since other modules might also put variables into the local variable collection for maintenance. So I have to retrieve variables from Metrics module for proper management for example re-initialization.
The problem is when I call snt.get_variables_in_module(metrics), it returns nothing. I just wonder is it the behavior by design or it needs some enhancement?
Thanks a lot.

@mingyr snt.get_variables_in_module takes a collection kwarg, which has the same options as tf.get_collection. If you change your code to snt.get_variables_in_module(metrics, tf.GraphKeys.LOCAL_VARIABLES) you'll get the same result as from the direct get_collection call.


Crashes on Google Pixel 2 and 2 XL

Recently the app crashes on image capturing but only on Google Pixel 2 and Pixel 2 XL devices.
Crash report shows the following:
java.lang.NullPointerException:
  at ....Camera2BasicFragment$ImageSaver.run (Camera2BasicFragment.java:935)
  at android.os.Handler.handleCallback (Handler.java:873)
  at android.os.Handler.dispatchMessage (Handler.java:99)
  at android.os.Looper.loop (Looper.java:193)
  at android.os.HandlerThread.run (HandlerThread.java:65)

This is line 935 (ImageSaver class):
ByteBuffer buffer = mImage.getPlanes()[0].getBuffer();
I've tried a few things but I'm unable to test as I do not have a Pixel 2 device. Any suggestions?
Update (06-Nov-2018):
So the logical explanation is that the image being returned from the ImageReader is somehow null:
@Override
        public void onImageAvailable(ImageReader reader) {
            mBackgroundHandler.post(new ImageSaver(reader.acquireLatestImage(), mFile));
        }

Some other posts suggest that the ImageReader format may not be supported. This is the format:
mImageReader = ImageReader.newInstance(largest.getWidth(), largest.getHeight(),
 ImageFormat.JPEG, /*maxImages*/2);

But I can't imagine why JPEG would be an unsupported format.

This sample has been migrated to a new location where we can accept Pull Requests (check README for more information).
As recommended by GitHub, we are closing all issues and pull requests now that this older repo will be archived.
If you still see this issue in the updated repo, please reopen the issue/PR there. Thank you!


soyosource_modbus need a name now

I got the following error:
soyosource_modbus: [source /config/esphome/soyosource.yaml:37]

  'name' is a required option for [0].
  - [source /config/esphome/soyosource.yaml:37]
    {}

There is this name used for?
Can i change
soyosource_modbus:

to
soyosource_modbus:
   name: ${name}

Would be nice if the readme will be updated :)

The name is used as prefix for the log messages. If you have multiple inverters the prefix/name is helpful to distinguish between the different devices.

I've added a default (name: "unnamed") now to avoid the breaking change and disturbing more people. :-)


Added DIS - Study abroad in Scandinavia in Copenhagen, Denmark

website of DIS: https://disabroad.org/
domain: dis.dk

@BLueders According to https://disabroad.org/dis/

DIS is a non-profit study abroad foundation established in Denmark in 1959, with locations in Copenhagen and Stockholm. DIS provides semester, academic year, and summer programs taught in English, and offers high-impact learning experiences for upper-division undergraduate students from distinguished North American colleges and universities.

It means that all students who are somehow affiliated with DIS, are also undergraduate students from distinguished North American colleges and universities, and they must have their email addresses assigned them by their respective universities.
Therefore, I can not satisfy this request, because DIS is not a college, nor a university, and also the students have university email addresses from other universities, i.e. they effectively don't need to use DIS email address for requesting their free JetBrains license. If DIS teachers need to use free educational licenses for JetBrains tools, please send an email about it to<EMAIL_ADDRESS>and we'll suggest another option to do it other than adding DIS to this repository.
Thank you for the understanding.


elm-live taking 70% of CPU also when idle

Recently I am getting the problem of elm-live getting around 70% of CPU resources also when is doing nothing.
I have the same issue on two macs. This problem appear around one month ago.
MacBook Pro - OS El Capitan
Node v.7.6.0
elm-live v.3.4.0

Hey! I merged your PR and tested everything and it looked good! Thank you so much for looking into this and figuring out what was going on. :)


Icon Styling with Masque

Request from CurseForge comments:

aujkis said: Possible support to style loot icons with Masque?

This is a little tougher than I thought since Masque is meant for styling buttons not just textures. So I started making progress but since the textures I am showing are using the FileDataId of the items and currencies, I don't have access to the file path which is required to set a texture on a button.
I tried a few different things and didn't make much progress. I'll come back to this later.


Bundling doesn't work at all

Hey guys,
maybe I'm just beeing dumb, but I can't get bundling to work at all. Here's what I do:
First, I updated SystemJS to v0.19.17 using jspm dl-loader. I then updated the bundler to v0.3.1 using node. After that, I ran into issue #66 which I got solved. Bundling should be working fine and there are no errors when I run gulp bundle. There is a lot of stuff, that is built in the dist folder: My bundles and all the components.
My bundles.json looks like this:
{
  "bundles": {
    "dist/app-build": {
      "includes": [
        "[*.js]",
        "*.html!text",
        "*.css!text"
      ],
      "options": {
        "inject": true,
        "minify": true
      }
    },
    "view-bundle": {
      "skip" : false,
      "htmlimport": true,
      "includes": "dist/*.html",
      "options": {
        "inject": {
          "indexFile": "index.html",
          "destFile": "dest_index.html"
        }
      }
    },
    "dist/aurelia": {
      "includes": [
        "aurelia-framework",
        "aurelia-bootstrapper",
        "aurelia-fetch-client",
        "aurelia-router",
        "aurelia-configuration",
        "aurelia-animator-css",
        "aurelia-templating-binding",
        "aurelia-templating-resources",
        "aurelia-templating-router",
        "aurelia-loader-default",
        "aurelia-history-browser",
        "aurelia-logging-console",
        "aurelia-auth",
        "github:paulvanbladel/aurelia-auth",
        "bootstrap",
        "bootstrap/css/bootstrap.css!text"
      ],
      "options": {
        "inject": true,
        "minify": true,
        "rev": true
      }
    }
  }
}

Both on localhost and on a remote host (running Apache), there are round about 40 requests made, most of them concerning the components (which are still loaded from the src folder). Am I doing something the completely wrong way? Am I just misunderstanding the concept of bundling?
Thanks for your help!

Can you share your output. Otherwise, we can't determine what's going wrong for you. Thanks.

Sorry. I can of course share any output you need... which part is interesting?
In the console, there are some debugging infos, the "Network" tab of the DevTools shows 493 XHR requests with the unbundled version (!!) and 35 XHR requests with the bundled one.
I made a screenshot of the both the Network tab and the Console output of the bundled version:

Network-Tab: http://grab.by/NIKU
Console-Output: http://grab.by/NIL8

The console output for the unbundled version looks the same, there are just those hundreds of requests.
Here's the top of the request list: http://grab.by/NILy
And here's the bottom: http://grab.by/NILu
Shouldn't the bundling reduce the number of requests to some very few? All the ViewModels (js files) should be included in the app-build.js or am I getting this wrong?
Thanks for looking into it!

So, I did some tests now. And I read the SystemJS documentation... ;-) As far as I can see, it's just not working properly. SystemJS still loads the modules from their raw files from either the dist folder or - in some cases - even from the src folder. This is completely indepentent from changes I make to my bundle configuration (minifying or not, injecting or not, and so on).
One strange thing I noticed: There is the view-bundle, which isn't created at all. Both the index.html and the dest_index.html do exist, but they look completely the same. Also, the main config.js isn't touched during bundling or unbundling.
Any ideas on this?

Okay now. Trying to help me out by myself... ;-) Getting closer.
I found a blog post, where it states "[*/**/*.js]" as a useful wildcard for subdirectories. I tried to add that, and it changes something. The system loader does now look for the subfolders in dist/src/subfolder/.... It can't find them there, because they are built in the correct place (dist/subfolder/...) - how to make it look at the right place?
Also, the files are not minified into the app-build.js, as they should. Why not?

Um. Before you do anything else try what I've described here:
http://www.foursails.co/blog/aurelia-bundling/

Thanks, I'll try that tomorrow!

@moay Is there any update on this?

Closing since there was no response from the OP after Matt's recommendation.


Add /metrics api.

Codecov Report

Merging #125 into master will increase coverage by 0.53%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master     #125      +/-   ##
==========================================
+ Coverage   66.05%   66.58%   +0.53%
==========================================
  Files          18       18
  Lines        3956     3956
==========================================
+ Hits         2613     2634      +21
+ Misses       1176     1155      -21
  Partials      167      167

Impacted Files
Coverage Δ

server/service/util/dependency.go
54.51% <0%> (+2.66%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 4d6c65d...015638d. Read the comment docs.

Coverage increased (+0.5%) to 70.921% when pulling 015638dc4bb767dce6597ee288bc1f17e9f2a2e9 on little-cui:master into 4d6c65daa8f7afe96825e51e737265e682ae502f on ServiceComb:master.


[wip] Added CPI configuration

 Write user case for cloud provider UI part in Velum

right now it tries to behave the same as the other cloud providers like ec2 and azure. meaning it expects a dedicated view for that in the worker-bootstrap page. similar to https://github.com/kubic-project/velum/blob/master/app/views/setup/worker_bootstrap_ec2.html.slim
but cant find /srv/velum/config/data/openstack-types.json when clicking next.
so i think you need to add an exception to https://github.com/kubic-project/velum/pull/451/files#diff-96d7538446a97258fe291a027a04c6baL57 regarding openstack, to not try to render a cloud worker view

right now it tries to behave the same as the other cloud providers like ec2 and azure. meaning it expects a dedicated view for that in the worker-bootstrap page. similar to https://github.com/kubic-project/velum/blob/master/app/views/setup/worker_bootstrap_ec2.html.slim

You are right. We need to ask public cloud team what we should deliver in this pull request. But I'm aware that not all the options are functional, I was just playing around. I think it will only be EC2 and Azure for now but I also make it ready for openstack too on behind the scenes.

I'm wondering if these openstack fields shouldn't be in a dedicated page. I guess when the openstack integration is done, this will be handled. Anyway, only EC2 and Azure are enabled atm.

I'm wondering if these openstack fields shouldn't be in a dedicated page. I guess when the openstack integration is done, this will be handled. Anyway, only EC2 and Azure are enabled atm.

Yes, you are right OpenStack fields are dedicated only for openstack driver, Azure, AWS and GCE have different fields.

Yes, you are right OpenStack fields are dedicated only for openstack driver, Azure, AWS and GCE have different fields.

What I wanted to know if it should be in a dedicated page as it was done for EC2 and Azure or if it's in the same page as shown in the screenshots. Let's wait for some feedback.

Confusions were cleared. I'm going to proceed with the expected changes.

@mjura This is good to be tested again.

Public Cloud environments will be automatically detected; our test images already provide appropriate selection of the UI workflow, by injecting the cloud:framework pillar at boot in EC2, Azure. There's no need for a dropdown to select, for example, EC2 as a framework.

I agree with @bear454 . By adding this dropdown an additional (unnecessary) step would get added for the user. Let's remove the dropdown step of the UI unless there's an important reason to keep it.

@aosthof @bear454: This change is about enabling Kubernetes Cloud Provider Integration driver, it doesn't have anything common with Public Cloud integration yet. If you would like to get more details please check it out https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/

@mjura I've reverted the rename that I did before. Please review this again.

@vitoravelino @mjura i have added the cloud provider to the pillars api in a separate commit, otherwise salt wouldn't set the pillars

@vitoravelino @mjura i have added the cloud provider to the pillars api in a separate commit, otherwise salt wouldn't set the pillars

Thanks!


feat: update version

Bump version number to 1.11.0. This version contains all of the types for the car improvements release of 27-01-2021.

:tada: This PR is included in version 1.11.0 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:

:tada: This PR is included in version 1.11.0 :tada:
The release is available on:

GitHub release
npm package (@latest dist-tag)

Your semantic-release bot :package::rocket:


New external endpoint doesn't allow Manage access to it.

Bug description
Add new external endpoint in json file and restart portainer, it shows in interface, but when you want to Manage access to this new endpoint, it gives error:

Failure
Unable to find an endpoint group with the specified identifier inside the database

And it doesn't load Access management interface part.
Expected behavior
After adding new external endpoint to allow Manage access to it even without default group. Or set default group to new external endpoints.
Steps to reproduce the issue:

Add new external endpoint in endpoints.json
Restart portainer
Go to endpoints list
Try Manage access to newly added endpoint.

Technical details:

Portainer version: 1.19.1
Docker version (managed by Portainer): 17.12.0-ce
Platform (windows/linux): linux
Command used to start Portainer (docker run -p 9000:9000 portainer/portainer):
docker run --detach=true --stop-timeout 60 -p "9000:9000" portainer/portainer
--external-endpoints /etc/portainer/endpoints.json --sync-interval 60s
Browser: Firefox

Hi @Ubusubus
Can you show us an example of an endpoint that is defined in the external endpoint file?

Ok. I will check it

I encountered this issue, and it seemed to boil down to the fact that when --external-endpoints is used, Portainer does not automatically assign those endpoints to the "Unassigned" group (which it usually would). If you re-start your Portainer instance using the same data, but without the --external-endpoints flag, then you see those endpoints get added to the Unassigned group as they usually would. THEN.... from that point forward, even if you go back to starting Portainer with the --external-endpoints flag..... the Unassigned grouping will remain and everything should work correctly again.
This issue essentially prevented me from using LDAP integration, until I figured out that the endpoints needed some sort of group assignment for user auto-creation to happen at all.

I have recreated this by simply trying to manage access on an external endpoint which was not assigned to a group and got this error

Reproduced on 1.23.2

Closing this issue as we're deprecating the external endpoint feature in 1.24.0 and it will be removed in 2.0.
See https://github.com/portainer/portainer/issues/3832


How to disable response validation?

Hello,
Right now response is mandatory parameter for validation. My attempts to remove it or suppress it doesn't work.
Any suggestions will be highly appreciated!
Thank you

Yes, I use version "nestjs-typebox": "2.6.1"
But any my attempt to omit it failed. Validator shows error in this case. :(

Interesting, so it sounds like you mean a runtime error, not a type error?
I'll take a closer look at this, but again I'm using this package in several projects in production so I'm fairly certain things are working under normal circumstances. Perhaps one thing you can try in the meantime is to use Type.Any() for the response schema. It could also be that the version of type box you are on is incompatible. There have been a lot of changes in the library over the past month or two and so this project has been pinning the type box version until things stabilize. You can expect an update to the latest version sometime this month.

Thank you!
The working solution for 2.6.1 is
 response: { schema: Type.Any(), stripUnknownProps: false }


Stop ScopeChain lookups when first scope is local

A local scope should stop the lookup from proceeding along the chain. This
was working in all cases except when the very first scope in the chain was
local.

I only just realised that this has a rather important effect on how default parameters to macros are evaluated. Without this pull request, they are able to see variables that are in the caller's scope and with this pull request they are not.
Which is the correct behaviour?

Hi, thanks for the merge but did you see my comment from Aug 22? This change breaks backward compatibility - we reverted it from our internal fork because we had people relying on the old behaviour.


[Feature] Issue and pull request related interactions

Description
In issue and pull request charts, when clicking on a certain month data point we can show the data of the certain month.
Like when clicking on the 2021-11 data point in the issue chart on the opens line, we can jump into a new URL like https://github.com/hypertrons/hypertrons-crx/issues?q=is%3Aissue+created%3A2021-11-01..2021-11-30
Or when clicking on the 2020-11 data point in the pull request chart on the merges line, we can jump into a new URL like https://github.com/hypertrons/hypertrons-crx/pulls?q=is%3Apr+merged%3A2021-11-01..2021-11-30
Through this, we can introduce more interactions into Hypercrx and the data charts are not only for observing but also provide quick entrance into detailed data.

Looks similar to a previous issue:

#430
I made this one a good first issue, anyone wants to take it? Can refer to the issue I mentioned above.

@wxharry Yes, quite similar, we can design a mechanism to show data for all possible chart interactions.

@wxharry Does #430 work now? I find that in my Hypercrx, clicking on activity or influence chart does not lead to a new URL.

Yes. What version are you using? There was a known bug on that feature, clicking is useless on English version. We have fixed it, but might haven't publish that version yet.

Thanks, I am using Hypercrx with the English version, the function works fine after switching to the Chinese version.

/self-assign

closed via #524


Template seems to break when using hyperref

The template seems to be doing some hacks with the table of contents, so if we comment out \tableofcontents, hyperref works.
Potential fix at https://github.com/mchladek/ucetd-latex/blob/master/ucetd.cls#L302-L322
Thanks to @suhailrehman for reporting and finding the potential fix.

Before fix:

After fix:

I will consider merging after verifying it meets the spec.

Relevant: https://github.com/latex3/hyperref/issues/121


Set more specific batch bundle action in audit event when possible

Is your feature request related to a problem? Please describe.
In #1813, we introduce a change to start logging batch/transaction bundle requests via an audit event with action "E".
However, for bundles which are all reads, all creates, all updates, or all deletes, we should be able to be a little more specific.
Describe the solution you'd like
If the bundle is all reads, set "R".
If the bundle is all creates, set "C".
If the bundles is all updates, set "U".
If the bundle is all deletes, set "D".
Else, set "E".
Describe alternatives you've considered
If we decided to log each individual entry for both batch and transaction bundles (see #1816), then this task becomes a lot less important, because we would already have the info from each individual entry from within the bundle.
Additional context
Currently, we track how many reads, creates, and updates we have for each bundle.  But for some reason we don't track how many deletes.  That should be added under this work item so that we can set the action to "D" when appropriate.

Could be done together with #1816

Verified the different "action" tags of "R", "C", "U". "D", and "E" are in the audit record.  Noting that a PUT no matter the return code (201, or 200) is marked as an Update.  Will create a new issue for this in case we want to address that.


app launches from Terminal but not Finder

It is possible to launch the app from the terminal by running
./Kolibri.app/Contents/MacOS/Kolibri

However it is not possible to launch from the Finder.
@kollivier says:

Checking kolibri.txt in the logs, the cherrypy server terminates about 10 seconds after starting up. I don't see any errors before the shutdown though.

Tested on Mojave 10.14

Still extant on the 0.13.0b2 release.
Asks for git to be installed on first run:

Fails to start the server:

But it can be started by executing ./Kolibri.app/Contents/MacOS/Kolibri:


Make the default version search pattern for tags more liberal.

In our default version tagging scheme we prefix all tags with "version/".
So instead of "1.2.3" as a version tag, we have "version/1.2.3".
This commit makes the default version pattern matcher ignore every prefix that does not contain numbers.
So "banana/1.2.3" would also count as a version tag even though we have no idea why people would want to prefix their version numbers with banana's.
A few comments:

It would be even nicer if that pattern was configuratble - so part of the jgitver.config.xml
VersionNamingConfiguration.java should probably have constructors that takes Pattern objects rather than strings.
These defaults should probably be located there as well, so you could have a constructor that takes no patterns but instead uses the default built-in patterns.

None of the above is a big deal, it's a pretty nice codebase overall.

@randakar I agree that making it configurable is the way to go to cover this need, I'll create issues in both jgitver & its plugins to adapt.
Thank you for reporting this RFE.

Thanks. It's not a really big deal, I'm just not looking forward to rewriting all version tags in all of our repo's just to automate the versioning ;-)

@randakar It is not a big change and it is very valuable (see #20). Expect it in the coming days ; do not waste your time changing your tags. For the moment you can just apply your changes temporary to both jgitver & jgitver-maven-plugin until I release the feature.

Thanks :)
We can wait a little while, so far this is only one experimental branch we're playing around in.
Thanks for the quick feedback!

(Leaving this open mostly as a thread watch)

feature implemented & integrated via resolution of #20.


NTR: allied health profession/practitioner role

As I mentioned, we have a need to have allied health profession (practitioner)role
This is less defined and we currently are working on this branch in MAxO (https://github.com/monarch-initiative/MAxO/issues/187).
Allied health professional defined by NCIt is ' Professional health-care providers who are not physicians, especially medical assistants, technicians, and therapists but not nurses.' Although I was thinking of a definition more like 'a professional who provides a range of diagnostic, technical, therapeutic, and support services in connection with health care. This is care provided by health care practitioners who  are distinct providers from dentistry, nursing, medicine, and pharmacy health care professionals.'
I was thinking of terms like

[ ] occupational therapist role
[ ] speech therapist role
[ ] speech language pathologist role
[ ] dietician evaluation (registered dietician?) role
[ ] audiologist evaluation role
[ ] physical therapist role
[ ] phlebotomist role
[ ] rehabilitation specialist role
[ ] respiratory therapist role
[ ] social worker role

but this isn't entirely fleshed out yet.
Let me know if you are interested in discussing this for inclusion into OMRSE.
Thanks.

Is there any way we can make it easier to help you add these?


[Fix]contact link locale

Coverage remained the same at 83.358% when pulling 85c3f6cd780ccc6c755dd37ec373012bcd5972a2 on ShinjiTanimoto:locale into 2b16ebef10edd31aad2925754418d3215706feae on shirasagi:master.


How to use png favicon files?

I have a file named favicon.png and want to use it with serve-favicon.

Hi @techyrajeev you cannot with this module. This module is exclusively for serving the "default, implicit favicon", which is GET /favicon.ico. For additional vendor-specific icons that require HTML markup, additional middleware is required to serve the relevant files, for example serve-static.


[automation] Update links on whats-new doc

Hyperlinks to the mentioned repositories added.

#sign-off


Thank You

Just want to thank you for the script and the only way I know is to open an issue 🙂. The script worked great for us. Really simple to setup and use. Please close the issue once you see it.

Glad I could help.

Please feel free to spread the word a bit, if you don't mind.


Implement Matchdata#to_a and #captures

@seven1m The problem with adding that assertion is it'll fail when there are no groups in the regexp. For example:
$ bin/natalie
nat> /..\d+\d/.match("THX1138.").captures
Assertion failed: (start < m_region->num_regs), function array, file match_data_object.cpp, line 6.

That case doesn't seem to be covered by ruby/spec but MRI behavior is to return an empty array:
$ irb
irb(main):001:0> /..\d+\d/.match("THX1138.").captures
=> []

The MatchDataObject::array() function is only called internally with either 0 or 1 as the start... an alternative could be to make it private to make it harder to accidentally misuse?

Ahh, very good. I misread the function, thinking it would segfault if the start was too high. I see now it just returns an empty array, which is great. I added a test and removed the assertion in cf8b4582cc5144f1f253f764324469e1f8cea34c. <3


Make the lib work on Android

The lib doesn't work on android unless I disable the ssl cert verification. I'm not really good at this, but if this solution looks stupid, can you suggest any workaround to make this work on android without disabling security features?

How did you get it to work with this change?
If i do that the openssl lib throws an divided by zero error in the bn_asm.c file at line 287. Which seems odd to me.

Hi, can you check the version of openssl in the submodule, it works perfectly fine for me with OpenSSL 1.0.2k

@kebbbnnn
strange i have the same version

Hey @kebbbnnn, @drubinstein, isn't it a fairly serious security risk to disable SSL verification? Doesn't that kinda defeat the purpose of SSL--or is there something I'm missing here?


Calling show crashes (after VTK update it seems)

My code using Vedo was working correctly last week. I updated vtk today (https://pypi.org/project/vtk/). Since then my code crashes. Running various Vedo examples step-by-step (see below), the execution crashes when calling "show" (Segmentation fault (core dumped)).
When running my code from PyCharm, I get the message :
Process finished with exit code 139 (interrupted by signal 11: SIGSEGV)
Examples I tried to run :
https://github.com/marcomusy/vedo/tree/master/examples/other/icon.py
https://github.com/marcomusy/vedo/tree/master/examples/basic/sliders3d.py
Thank you,
Eric

On 2021-11-08 15:49, Marco Musy wrote:

Hi Eric, thanks for reporting, yes it's a bug in the latest vtk
release... I will make a new release today to fix that!

Thank a lot for being so prompt at answering.
Best regards,
Eric

--
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub [1], or unsubscribe
[2].
Triage notifications on the go with GitHub Mobile for iOS [3] or
Android [4].
Links:
[1] https://github.com/marcomusy/vedo/issues/518#issuecomment-963234138
[2]
https://github.com/notifications/unsubscribe-auth/ALSYTPCJ2JZJZLLGBGPGWITUK7PPZANCNFSM5HSVH5TQ
[3]
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
[4]
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub

Hopefully it's now working after upgrading:
pip install vedo -U
Let us know if you encounter any problems,
Cheers
Marco

Indeed, it works again. Thanks a lot.
Global thanks to the whole team of contributors also.
Best,
Eric
On 2021-11-09 02:11, Marco Musy wrote:

Hopefully it's now working after upgrading:
pip install vedo -U
Let us know if you encounter any problems,
Cheers
Marco
--
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub [1], or unsubscribe
[2].
Triage notifications on the go with GitHub Mobile for iOS [3] or
Android [4].
Links:
[1] https://github.com/marcomusy/vedo/issues/518#issuecomment-963720938
[2]
https://github.com/notifications/unsubscribe-auth/ALSYTPHARE3KUNFXPFA5DN3ULBYL3ANCNFSM5HSVH5TQ
[3]
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
[4]
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub


[SPARK-24846][SQL] Made hashCode ExprId independent of jvmId

What changes were proposed in this pull request?
Made ExprId hashCode independent of jvmId to make canonicalization independent of JVM
(Please fill in changes proposed in this fix)
Override hashCode (and necessarily also equality) to depend on id only
How was this patch tested?
(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Created a unit test ExprIdSuite
Ran all unit tests of sql/catalyst
Please review http://spark.apache.org/contributing.html before opening a pull request.

Can one of the admins verify this patch?

Can one of the admins verify this patch?

Can one of the admins verify this patch?

ok to test

Test build #93240 has started for PR 21806 at commit 68d6f19.

@gvr can you clean-up the description somewhat? It currently also has part of the template in it.

Updated description, thanks @hvanhovell

The change looks fine. However I'm wondering that have we have chance to compare hash code between expr ids from different jvms?

@viirya this current change is only useful when you compare canonicalized plans created on different JVMs. This has come up when we tried to detect changes in plans over spark versions (plan stability) and when people are writing to caches used by multiple clusters.

@hvanhovell Got it. Thanks for your explanation. LGTM.

LGTM too

Merging to master. Thanks!


update milvus to 2.3.x

Milvus has released a new version with GPU support:
https://milvus.io/blog/unveiling-milvus-2-3-milestone-release-offering-support-for-gpu-arm64-cdc-and-other-features.md
As we have an Nvidia A10 GPU on our production server that we want to use for this, can you update milvus and test to make sure the existing index does not get wiped out?
It would be good to have to have a flag in the docker file to activate or deactivate this so that it can also be used on servers without GPU support.
Thanks!

nevermind. I restarted the containter and it built with the latest 2.3.2

false alarm. It was just the ATTU interface that reported the wrong version


Bug-Fix in Polynomial-Features

While working on #366 I realized that the PolynomialFeatures transformer does not respect the passed parameters correctly. The PR fixes this and provides a test

Opened https://github.com/dask/dask-glm/pull/72 in relation to some of the test failures here. Usage of dask.set_options and get= in dask-glm are causing errors to be raised here.

I'll rebase hopefully, later today

@datajanko it should be okay to rebase now, the errors showing up here have been fixed in master. Thanks for your patience.


[Bug]Ammo packs bought on flea have 0/x ammo in them and cannot be opened.

Issue is for: Server
Expected Result: Have x/x ammo not 0/x ammo after buying packs off the flea.
Actual Result: The ammo backs have 0/x ammo and cannot be opened after buying off the flea.
Description: You buy an ammo pack off the flea and it comes with no ammo inside and the option to unpack is blacked out and cannot be clicked, but buying ammo packs off the traders directly works just fine and you have the ammo in the pack and can be unpacked.
How to reproduce:

go to flea
buy any ammo pack
go to stash
check ammo pack

Attachments

server-2024-01-15.log (103 KiB)
65a5be420000a569af943730.txt (223 KiB)
LogOutput.log (8 KiB)
2024.01.15_23-22-52_<IP_ADDRESS>.28375 traces.log (44 KiB)

Originally written by Schizo1940

possibly linked with issue #297 as ammo boxes not being consumed

Originally written by naterade

Looks like the recent move to addItemToStash doesn't call hydrateAmmoBoxWithAmmo() like addItem() does.
I'm going to work on an update. I'll put out a PR if no one beats me to it.

Originally written by Spoony

Fixed with: https://dev.sp-tarkov.com/SPT-AKI/Server/commit/e1a8f26507a3e03914b7d6c129300e066fa4a60e

Originally written by chomp

Tested with latest build as of 0842 EST 1/16/2024 with commit fbee59fede. Can confirm issue can be closed

Originally written by naterade


Document new Start-MigrationUser parameters

Providing documentation for the new CompleteAfter and StartAfter parameters for the Set-MigrationUser cmdlet.  Also, this PR contains improvements to the CompleteAfter/StartAfter description in other files (Set-MigrationBatch and New-MigrationBatch), adds a description of what the Update flag does (for Set-MigrationBatch), and adds a note about the relationship between settings that are applied to a MigrationUser versus settings that are applied to the MigrationBatch that contains them.

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

exchange/exchange-ps/exchange/move-and-migration/Complete-MigrationBatch.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/move-and-migration/New-MigrationBatch.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/move-and-migration/Set-MigrationBatch.md
:white_check_mark:Succeeded
View (exchange-ps)

exchange/exchange-ps/exchange/move-and-migration/Set-MigrationUser.md
:white_check_mark:Succeeded
View (exchange-ps)

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


If a directory and a file have the same basename only one gets processed

Take a look at Semantic-UI's tasks folder: they have a build.js main task, that requires all the individual partial builders from the build directory, both are located at the same path.
I'm building something similar, but in a cleaner why with the help of this library. The problem is that since both objects have the same basename only the directory gets processed, hence the build.js file never gets loaded.
A possibile solution to this problem would be to overwrite the previously defined key of the returned hash, I think that processing all the files is more important. In my case none of the files located in the build directory is even actually exporting anything.

Have you tried setting duplicates option to true as it's described in Readme?

Setting duplicates to true works. I must have missed it, thanks!


implement get_mut_or_default

Fixes #561
Checklist

[x] I've added tests for all code changes and additions (where applicable)
[ ] I've added a demonstration of the new feature to one or more examples (didn't find any relevant examples)
[ ] I've updated the book to reflect my changes
[x] Usage of new public items is shown in the API docs

API changes
New function: GenericWriteStorage::get_mut_or_default for Default component types. Not breaking.

This change is

Squashed & pushed with relevant modifications.

Marked that it will panic if the entity is dead. Another option is to return an Option, which'd make the usage slightly less ergonomic, but in most cases still nice enough.

^ Opened a separate PR for that. Pick one.


Can't build iOS

I'm following your iOS Setup, and at the moment I create the app delegate I have the following error
../iosApp/iosApp/AppDelegate.swift:33:17: error: cannot find 'GIDSignIn' in scope
GIDSignIn.sharedInstance.handle(url)
^~~~~~~~~
Can you help me figuring it out?

@gabrieloterostride Please make sure you add Google Sign in Swift side as well.
I shared in this issue: https://github.com/mirzemehdi/KMPAuth/issues/6

It worked! thanks!


Version 5.15.11 is not available

platform is amd64 debian bookworm
just run
aqt list-qt linux desktop

to reproduce

Hi, please see:

#572
#636
#668


Cleanup the value mappers

This started with a big ugly switch.  Breaking it out into smaller classes.

If you remove any one of these mappers, tests will fail.  Put them back in, and the tests go green.
So I'm confident that the behavior and logic is maintained through the data mapper tests, even though each class does not have its own tests.


Long running scripts get duplicated

bitbar v2-beta10.
If I run the Refresh All action all my long-running scripts get duplicated. There's no problem with short-lived scripts.
Am I supposed to kill the old processes by myself or something?

me too - i have two timer plugins up right now.

you could set an environment variable at the start of your script and clear it at the end. then in your script you can check if it's set and exit your script early rather than running it again.
ideally, this sort of check would be done in BitBar but at least it would be a workaround.

@gingerbeardman thanks for your suggestion, although I'm not sure it would actually work. It would ensure that only the old version of the script is running instead of both the old and the new, but I want the new version to be running, that's why I'm refreshing it in the first place.
I think simply by incrementing that environment variable on the new script, and checking if its value changed on the old, and in that case exiting, would do the trick.
If v2-beta10 didn't have any other deal breakers for me I would think about implemented this.

There are possibly multiple scenarios that would cause a plugin to duplicate, but I had one plugin that ran every 5 seconds and it would get duplicated, and changing it to 10 seconds fixed that.
I have another plugin that runs every 1 second, but it doesn't get duplicated, so as I said, there are probably several factors at play.

I have the same problem.


Update podspec file to 1.3.4

Update podspec file to 1.3.4
You cocoapod file is using 1.3.3

Done


DRAFT: An imgix tile source

https://www.imgix.com/
I think we are leaning against using this for the use case we developed it experimentally for. For the 100MB+ TIFF original files we have, it's hard to make imgix latency good. imgix isn't really designed for giant-original use case, although it does work fine if slowly with them. (imgix does have a a 512MB max for source file size though).  And the options for non-public/secured images are somewhat limited on imgix.
However, I think this OSD tile source actually works fine, within the limitations of the imgix service itself. (which is a pretty good and well-priced service when you are dealing with less insanely large source images).
It does not yet have any tests. I am not neccesarily proposing this be incorporated into OSD source. In fact, without being in OSD source, it's easy enough for an end-project to just load this
single file into their JS, and it'll work fine with OSD. (Which is what we were doing for testing).
I am mostly submitting it in case this work is of interest to someone else.  I'm myself probably done working with it at this point. Although I think this tile source code submitted here is actually working fine  and production-ready.

Beautiful! Thank you for sharing this.
I'm thinking we probably shouldn't include this as part of OSD, since Imgix is a proprietary service. It would be great to have it published somewhere as a plugin, though! @jrochkind Any interest in publishing it under your GitHub account? Or perhaps @imgix would be interested in publishing it under their account?

Honestly, I'm probably not going to be using it,  and have no interest in implying any maintenance of it, or really doing anything further with it at present.
I don't see why you can't include it (or publish as a plug-in yourself under @openseadragon) just because it points to a proprietary service. All the code here is open source.  If I actually wanted to use it, I might argue harder to convince you of this, but I currently have no horses in this race. :)
Are there examples of other third-party plugins hosted elsewhere? Do you list them in OSD docs? If not, nobody's ever gonna find it or realize OSD can do it, heh.
If my plans change and we end up using this in production, my opinions on these various things my change. :)

Yes, we've actually got a great collection of plugins, listed on our front page:
http://openseadragon.github.io/#plugins
Anyway, I definitely want it to be available! I think it might be ideal if @imgix adopts it, but we can adopt it if need be. Either way, we'll be sure to point people to it.

Cool. I don't think any existing plugins are tile sources?  If you have a plugin listed that's a tile source, I'd mention in the "Supported Tile Sources" section at https://openseadragon.github.io/#examples-and-features that people should check the plugins for more (and/or just list them there in a plugin section, and/or in a separate 'tile sources' section on the plugins page).
That "Supported Tile Sources" section is where I always end up when trying to figure out "wait, what types of tile sources does imgix support?"

Yes, absolutely, good point!
I think I'll leave this PR here for now while we think about next steps. Thank you again for building it!

I'm still not planning on using this, but it occurred to me you might be able to get a better balance of performance by making the tile size a lot bigger than the current default. The same is probably true of the IIIF tile source, or anything that's asking for tiles to be generated on demand instead of accessing pre-generated tiles.

@jrochkind Good point... I'm generally suggesting 512 or even 1024 pixel tiles these days.
@jpstroop @azaroth42 I'm not familiar enough with the IIIF tile source to know... does it do any sort of choosing for how big to request the tiles? Should we update it to a bigger number? Looks like we might already be supporting 1024, so maybe it's all good?

The back-end server is literally making tiles on demand, so can support whatever tile sizes the front-end asks for. I am not completely sure how the front-end works, but I think I saw a tile-size configuration property.

IIIF has the following information that is processed for generating the tile requests:
"tiles": [
    {"width" : 512, "scaleFactors" : [1,2,4,8,16]}
 ],

With an optional height if the tiles are not square. So the server can advertise the appropriate size for the particular instance, or even image.  If it's not supplied the IIIF TileSource picks a sensible (hopefully) default.
HTH

@azaroth42 Great... sounds like we're already doing the right thing; thank you for confirming!

Interesting. I have no idea if the various IIIF servers we were trying to use allowed us to set this in the info response the OSD plugin would get. I do think a larger tile size would have improved responsiveness for the tests we were doing. It might be nice to allow this to be overridden on the OSD end.
But for now, we're not going with an IIIF server either where i am -- we're just generating straight static DZI.

I suggest that if you would actually use an override for IIIF tile size in OSD (and it sounds like you probably wouldn't), then a new issue would be the appropriate place to discuss that.

The back-end server is loaded on demand, and my needs are the same, but what I see now are basically static dzi local files. Did not find the example I wanted. Finally I saw a little partner who was related to my thoughts.


Registrar error : unable to inspect container

I'm using marathon(https://mesosphere.github.io/marathon/docs/native-docker.html) to run a docker container on a mesos(http://mesos.apache.org/) cluster.
I see the following logs in the registrar container:
 unable to inspect container: 7001ffcfebf4 json: cannot unmarshal number 1.6777216e+08 into Go value of type int64

There was some permission issue because of which docker rest apis were failing.


Manage autoscale 0 and no node cluster

Q
A

Bug fix?
yes

New feature?
yes

API breaks?
no

Deprecations?
no

Related tickets
-

License
Apache 2.0

What's in this PR?

Feature allowing NifiClusternot to define the nodes in the spec or to make them an empty list.
Fix to handle empty CreationTime in node states.
Why?

In the case of a cluster managed by autoscaling, we want to be able to make autoscaling go all the way down to 0. But in the case where autoscaling manages all nodes, we need to allow a cluster to be no node at all.
Additional context

When deploying clusters using chart helm, we don't want the replicas field of the NifiNodeGroupAutoscaler and the nodes field of the NifiCluster to be defined, to avoid rewriting them in the event of a chart update. To do this, these fields must be optional and therefore completely managed in this case (= no node).
Checklist

[x] Implementation tested
[x] Error handling code meets the guideline
[x] Logging code meets the guideline
[x] User guide and development docs updated (if needed)
[x] Append changelog with changes

Sorry, i meant to review this. I mentioned this as a limitation when i contributed this feature:
https://github.com/konpyutaika/nifikop/pull/89

It's not possible to deploy a NifiCluster with only autoscaled node groups. The NifiCluster CRD requires that you specify at least one node in the spec.nodes list. Do we want to support this? If so, we may need to adjust the cluster initialization logic in the NifiCluster controller.

So it's nice to see this added. Is there an example of a minimum NiFiCluster + NifiNodeGroupAutoscaler configuration in the nifikop docs?

I haven't really changed the documentation, I've just corrected a few things because it's still true. But I guess the minimum configuration will be a NifiCluster without node and a NifiNodeGroupAutoscaler without replicas fully controlled by an HPA.


"Globals" nav missing "sel" class on select/active

Description
Selecting the "globals" nav In the control panel sidebar navigation doesn't attach the class "sel" to the nested anchor element.
Steps to Reproduce

Go to "layouts" and create a new layout
Click "Globals" in the Control Panel Sidebar and inspecting the HTML source will reveal that the "sel" class is missing from the nav item

The example below is what the code should look like when the nav item is selected.

Globals

Disabling the Control Panel Nav plugin restores the correct behaviour to the sidebar nav for the Globals nav item.
Additional info

Craft version: 3.6.10
Control Panel Nav version: 3.0.15

Fixed in 3.0.16


Flutter Line chart intermittent rendering issue

Same Problem. I hope someone is looking into it. It has been showstopper for us.
Please check below full error trace.
I/flutter (25291): ══╡ EXCEPTION CAUGHT BY RENDERING LIBRARY ╞═════════════════════════════════════════════════════════
I/flutter (25291): The following assertion was thrown during paint():
I/flutter (25291): 'package:charts_common/src/chart/layout/layout_manager_impl.dart': Failed assertion: line 115 pos
I/flutter (25291): 12: '_drawAreaBoundsOutdated == false': is not true.
I/flutter (25291):
I/flutter (25291): Either the assertion indicates an error in the framework itself, or we should provide substantially
I/flutter (25291): more information in this error message to help you determine and fix the underlying cause.
I/flutter (25291): In either case, please report this assertion by filing a bug on GitHub:
I/flutter (25291):   https://github.com/flutter/flutter/issues/new?template=BUG.md
I/flutter (25291):
I/flutter (25291): When the exception was thrown, this was the stack:
I/flutter (25291): #2      LayoutManagerImpl.drawableLayoutAreaBounds (package:charts_common/src/chart/layout/layout_manager_impl.dart:115:12)
I/flutter (25291): #3      BaseChart.drawableLayoutAreaBounds (package:charts_common/src/chart/common/base_chart.dart:414:22)
I/flutter (25291): #4      _LinePointLayoutView.paint (package:charts_common/src/chart/common/behavior/line_point_highlighter.dart:418:30)
I/flutter (25291): #5      BaseChart.paint.<anonymous closure> (package:charts_common/src/chart/common/base_chart.dart:579:12)
I/flutter (25291): #6      List.forEach (dart:core/runtime/libgrowable_array.dart:278:8)
I/flutter (25291): #7      BaseChart.paint (package:charts_common/src/chart/common/base_chart.dart:577:38)
I/flutter (25291): #8      ChartContainerCustomPaint.paint (package:charts_flutter/src/chart_container.dart:381:11)
I/flutter (25291): #9      RenderCustomPaint._paintWithPainter (package:flutter/src/rendering/custom_paint.dart:521:13)
I/flutter (25291): #10     RenderCustomPaint.paint (package:flutter/src/rendering/custom_paint.dart:559:7)
I/flutter (25291): #11     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #12     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #13     _RenderProxyBox&RenderBox&RenderObjectWithChildMixin&RenderProxyBoxMixin.paint (package:flutter/src/rendering/proxy_box.dart:123:15)
I/flutter (25291): #14     RenderPointerListener.paint (package:flutter/src/rendering/proxy_box.dart:2629:11)
I/flutter (25291): #15     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #16     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #17     _RenderProxyBox&RenderBox&RenderObjectWithChildMixin&RenderProxyBoxMixin.paint (package:flutter/src/rendering/proxy_box.dart:123:15)
I/flutter (25291): #18     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #19     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #20     _RenderCustomMultiChildLayoutBox&RenderBox&ContainerRenderObjectMixin&RenderBoxContainerDefaultsMixin.defaultPaint (package:flutter/src/rendering/box.dart:2273:15)
I/flutter (25291): #21     RenderCustomMultiChildLayoutBox.paint (package:flutter/src/rendering/custom_layout.dart:361:5)
I/flutter (25291): #22     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #23     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #24     _RenderProxyBox&RenderBox&RenderObjectWithChildMixin&RenderProxyBoxMixin.paint (package:flutter/src/rendering/proxy_box.dart:123:15)
I/flutter (25291): #25     RenderDecoratedBox.paint (package:flutter/src/rendering/proxy_box.dart:1987:11)
I/flutter (25291): #26     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #27     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #28     _RenderProxyBox&RenderBox&RenderObjectWithChildMixin&RenderProxyBoxMixin.paint (package:flutter/src/rendering/proxy_box.dart:123:15)
I/flutter (25291): #29     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #30     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #31     _RenderFlex&RenderBox&ContainerRenderObjectMixin&RenderBoxContainerDefaultsMixin.defaultPaint (package:flutter/src/rendering/box.dart:2273:15)
I/flutter (25291): #32     RenderFlex.paint (package:flutter/src/rendering/flex.dart:931:7)
I/flutter (25291): #33     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #34     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #35     _RenderProxyBox&RenderBox&RenderObjectWithChildMixin&RenderProxyBoxMixin.paint (package:flutter/src/rendering/proxy_box.dart:123:15)
I/flutter (25291): #36     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #37     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #38     _RenderFlex&RenderBox&ContainerRenderObjectMixin&RenderBoxContainerDefaultsMixin.defaultPaint (package:flutter/src/rendering/box.dart:2273:15)
I/flutter (25291): #39     RenderFlex.paint (package:flutter/src/rendering/flex.dart:931:7)
I/flutter (25291): #40     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #41     PaintingContext.paintChild (package:flutter/src/rendering/object.dart:173:13)
I/flutter (25291): #42     _RenderSingleChildViewport.paint.paintContents (package:flutter/src/widgets/single_child_scroll_view.dart:522:17)
I/flutter (25291): #43     PaintingContext.pushLayer (package:flutter/src/rendering/object.dart:370:12)
I/flutter (25291): #44     PaintingContext.pushClipRect (package:flutter/src/rendering/object.dart:394:7)
I/flutter (25291): #45     _RenderSingleChildViewport.paint (package:flutter/src/widgets/single_child_scroll_view.dart:526:17)
I/flutter (25291): #46     RenderObject._paintWithContext (package:flutter/src/rendering/object.dart:2092:7)
I/flutter (25291): #47     PaintingContext._repaintCompositedChild (package:flutter/src/rendering/object.dart:128:11)
I/flutter (25291): #48     PaintingContext.repaintCompositedChild (package:flutter/src/rendering/object.dart:96:5)
I/flutter (25291): #49     PipelineOwner.flushPaint (package:flutter/src/rendering/object.dart:853:29)
I/flutter (25291): #50     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding&PaintingBinding&SemanticsBinding&RendererBinding.drawFrame (package:flutter/src/rendering/binding.dart:331:19)
I/flutter (25291): #51     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding&PaintingBinding&SemanticsBinding&RendererBinding&WidgetsBinding.drawFrame (package:flutter/src/widgets/binding.dart:701:13)
I/flutter (25291): #52     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding&PaintingBinding&SemanticsBinding&RendererBinding._handlePersistentFrameCallback (package:flutter/src/rendering/binding.dart:268:5)
I/flutter (25291): #53     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding._invokeFrameCallback (package:flutter/src/scheduler/binding.dart:988:15)
I/flutter (25291): #54     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding.handleDrawFrame (package:flutter/src/scheduler/binding.dart:928:9)
I/flutter (25291): #55     _WidgetsFlutterBinding&BindingBase&GestureBinding&ServicesBinding&SchedulerBinding._handleDrawFrame (package:flutter/src/scheduler/binding.dart:840:5)
I/flutter (25291): #59     _invoke (dart:ui/hooks.dart:209:10)
I/flutter (25291): #60     _drawFrame (dart:ui/hooks.dart:168:3)
I/flutter (25291): (elided 5 frames from class _AssertionError and package dart:async)
I/flutter (25291):
I/flutter (25291): The following RenderObject was being processed when the exception was fired:
I/flutter (25291):   ChartContainerRenderObject<num>#e456d
I/flutter (25291):   creator: ChartContainer<num> ← Listener ← _GestureSemantics ← RawGestureDetector ← GestureDetector
I/flutter (25291):   ← LayoutId-[<chartContainer>] ← CustomMultiChildLayout ← LineChart ← DecoratedBox ← ConstrainedBox
I/flutter (25291):   ← Container ← Column ← ⋯
I/flutter (25291):   parentData: <none> (can use size)
I/flutter (25291):   constraints: BoxConstraints(w=360.0, h=200.0)
I/flutter (25291):   semantic boundary
I/flutter (25291):   size: Size(360.0, 200.0)
I/flutter (25291): This RenderObject has no descendants.
I/flutter (25291): ════════════════════════════════════════════════════════════════════════════════════════════════════
I/flutter (25291): Another exception was thrown: 'package:charts_common/src/chart/layout/layout_manager_impl.dart': Failed assertion: line 115 pos 12: '_drawAreaBoundsOutdated == false': is not true.
E

@2intor did you manage to fix it?

Without additional information, we are unfortunately not sure how to resolve this issue. We are therefore reluctantly going to close this bug for now. Please don't hesitate to comment on the bug if you have any more information for us; we will reopen it right away!
Thanks for your contribution.


User transforms and matches

it would be nice if there was a document that indicated which fields are and are not supported for matching against, as it seems I can't just try arbitrary fields, I.E. match authenticated yes resulted in an error and caddy didn't start.
I can't seem to match against any roles. for example, I would like the following to work, and it doesn't:

transform user {
  match roles res-caddy-admin
  action add roles authp/admin
  ui link "whoami" /whoami target_blank
}

@lbreitk , what are the versions of the modules you are using?

I just downloaded @latest yesterday, before testing and opening this issue.

@greenpau any ideas?

@lbreitk , will check after work.

Thanks man! :)

@lbreitk , could you please attach your full config (pastebin works too)?
Also, could you please change to “action add role authp/admin”? (role vs roles)

Sure. I'm positive I've tried both combinations. I also tried things like matching the email or other claims, some of which worked fine, with the same action item, so I'm thinking it's the matcher, not the action. I'll attach configs when I have a chance

@lbreitk , what comes to mind is whether the token has both "role" and "roles" claims.


predict! and fixed issue with sparse arrays of importance

fixes #87

@nbokulich, looks like flake8 is mad

oops sorry @ebolyen in all my excitement I forgot to check. Fixed!


Remove condition on configmap permissions

Since the default leader election resource lock is "configmapsleases",
which uses both resources, the permissions for configmaps is needed for
any cluster, not just OCP 3.11 clusters.
See https://pkg.go.dev/sigs.k8s.io/controller-runtime/pkg/manager#Options.LeaderElectionResourceLock
Signed-off-by: Justin Kulikauskas<EMAIL_ADDRESS>

/cc @ycao56
Thanks for noticing this and identifying the cause so quickly! This basically just updates the iam controller's permissions to match the (working) permissions in the cert controller.
From my tests, a role created with permissions for resources that aren't available on a cluster is still valid. So we don't need a condition to only put the coordination.k8s.io/leases permissions on the role if it's in a k8s cluster with that resource (eg, not OCP 3.11).

/approve


[gfx90a][FP16] Disable Solvers that unable to comply the "ALT FP16" attribute

Simplify access to ALT attribute from ProblemDescription
ConvBinWinogradRxSf2x3, ConvBinWinogradRxSf2x3g1
ConvBinWinogradRxSf3x2
ConvAsm1x1U, ConvAsm1x1UV2
ConvAsmBwdWrW1x1, ConvAsmBwdWrW3x3
ConvHipImplicitGemmBwdDataV1R1, ConvHipImplicitGemmBwdDataV1R1Xdlops
ConvHipImplicitGemmBwdDataV4R1Xdlops
ConvHipImplicitGemmForwardV4R4Xdlops, ConvHipImplicitGemmForwardV4R4Xdlops_Padded_Gemm
ConvHipImplicitGemmForwardV4R5Xdlops
ConvHipImplicitGemmV4R1Fwd, ConvHipImplicitGemmV4R1WrW
ConvHipImplicitGemmWrwV4R4Xdlops, ConvHipImplicitGemmWrwV4R4Xdlops_Padded_Gemm
Not necessary for:

ConvBinWinogradRxS due to this reasoning.
ConvHipImplicitGemmBwdDataV4R1, ConvHipImplicitGemmV4R4Fwd, ConvHipImplicitGemmV4R4WrW as these support only FP32.

:warning: @JehandadKhan From now on, many Bwd and WrW Solvers are disabled for gfx90a by default. Therefore, perf-db tuning will skip them. In order to tune perf-configs for them, MIOPEN_DEBUG_CONVOLUTION_ATTRIB_FP16_ALT_IMPL=0 can be set in the environment. This is just FYI. I do not think that tuning of disabled Solvers is really required.

Ready for review!

This solver should also been disabled
ConvCkIgemmFwdV6r1DlopsNchw
list:ConvCkIgemmFwdV6r1DlopsNchwhttps://github.com/ROCmSoftwarePlatform/MIOpen/blob/037e006c19be0d810c5cf1ced53b02e10d99702d/src/solver/conv_ck_igemm_fwd_v6r1_dlops_nchw.cpp#L82


#16431 resolve driver properties

Closes #16431
Has limitations:

Resolve will happen once, after that resolved value will be used
Resolving happens only in connection configuration stage

Verified


Parallel forward kinematics & collision checking

Thanks for your awesome work on pinocchio.
I'm wondering if it's possible to perform batch collision checking? For instance, I would like to sample 1000 different poses for my robot model in a fixed workspace, and check % of them collide with the obstacle. This would be very helpful for performing sampling-based motion planning tasks.
Thanks!

Yes, it is. The easiest is to have one Data and GeomData object per threads.

@jmirabel thanks for the note! I wonder if this would induce the same speed for 1000 GeomData object vs 1 GeomData object?
For my use case I will be using the same GeomData, at 1000 different configurations

1000 GeomData ?!? how many cores do you have ?

I wonder if this would induce the same speed for 1000 GeomData object vs 1 GeomData object?

My experience of multi-threading tells me that you will not be 1000 times faster. You can expect to have more cache miss with several cores at the same time. I advice you to carefully benchmark you code. Sometimes running with 2 threads is actually slower...

I think @hzyjerry wants to do batch computations in similar way it is done in deep learning with GPUs.
A short answer to @hzyjerry is that Pinocchio is not currently done for batch computations, but very tailored for multi-{CPU,process} or parallel computations.
There is a roadmap on adjusting Pinocchio for batch computations, but we are unfortunately lacking of manpower for it.

@jcarpent yup that's the ideal function that will suit my use case :)
Closing the issue for now. It would be very exciting to see progress in this direction (hopefully someday I can also contribute).

@hzyjerry We will really appreciate your contribution then.

Hi Pinocchio team, do you guys have any idea regarding the speedup that one could expect from using batch simulation instead on parallel computing. By experience, batch simulation is usually much faster for small matrices (robots with less than 20 DOFs).

Can you describe what you call batch simulation ?

Yes sure, it consists in adding a extra dimension corresponding to the ID of the robot to every tensors used for physics calculus, and to perform computations on those tensors directly without looping over every individual system, in order to get the most out the low-level linear algebra libraries. For instance, the position vector (robot.nq, 1) becomes a position matrix (num_robots, robot.nq), or the mass matrix a 3D tensor (num_robots, robot.nq, robot.nq).

It is relevant for GPU indeed because of the presence of thread. For CPU, it is less obvious, memory alignment matters.

I'm currently porting a part of Pinocchio on GPU ... but still works to do.

@duburcqa Which function do you see the real gain of batching?
By the way, collision detection would be hard to batch ...

On collision detection, I second @jcarpent : it is hard to achieve. Yet, there are some stuff which can be done.

some of the specialized pairs can be batched.
the BVH traversal can use vectorisation as a mean for parallelization. I did some trial on this and obtained better results than what I expected. The result are driven by the vectorisation capabilities of the CPU. No idea if this translates to GPU.

For CPU, it is less obvious, memory alignment matters.

Adding an extra dimension to tensors contradict memory alignment ?

This won't require to support the notion of a batch

What do you mean ? It will be enable to leverage the computational power of a high-end GPU (e.g. RTX 3090 or Tesla A100) without batching but using GPU multi-threading ?

Which function do you see the real gain of batching?

I was thinking about speeding up time-consuming rigid body algorithms such as forwardKinematics, in order to simulate multiple identical and independent robots at once as fast as possible for reinforcement learning applications. Regarding stuffs like collision computations, it would awesome but it was not expected that much since it relies on other libraries.

For CPU, it is less obvious, memory alignment matters.

Adding an extra dimension to tensors contradict memory alignment ?

Pinocchio is highly efficient because all the computations are predictable by the CPU with a kind of continuous flow of operation. When working with large batch, you will penalize this aspect, because you will work on larger memory segments which might impact the catch and the memory loading.

What do you mean ? It will be enable to leverage the computational power of a high-end GPU (e.g. RTX 3090 or Tesla A100) without batching but using GPU multi-threading ?

It might be not needed to explicitly support the notion of batch for the standard code. You can be much smarter by indeed exploiting the GPU thread, but there is no magic here, we need to implement this dispatcher, which is hard to make it generic. Won't you think so @duburcqa?

I was thinking about speeding up time-consuming rigid body algorithms such as forwardKinematics, in order to simulate multiple identical and independent robots at once as fast as possible for reinforcement learning applications. Regarding stuffs like collision computations, it would awesome but it was not expected that much since it relies on other libraries.

Common on, forwardKinematics is one of the cheapest operation in Pinocchio (1 or 2 us for 40-dof legged robots).
You won't gain anything by using GPUs due to the data transfer!!!

Regarding stuffs like collision computations, it would awesome but it was not expected that much since it relies on other libraries.

Pinocchio and FCL are closely related.

Come on, forwardKinematics is one of the cheapest operation in Pinocchio (1 or 2 us for 40-dof legged robots).

That was just an example, basically it affects any rigid body algorithm. And regard 1us for 40dof system, it is still very slow compared to Nvidia Isaac Gym, which has a simulation-to-real time ratio above 5e4 (1e-7s to perform a single integration step).

If you use the code generation support of Pinocchio, you can really decrease these timings by 5 or even 10.

Yes this is an interesting feature, but it is hard to take advantage of it in Python since no JIT compiler support for Pinocchio for Pinocchio, and in the reinforcement learning it is unusual to pre-compile C++ by yourself for a specific application.

Yes, you can do JIT with Pinocchio !!!

Yes, you can do JIT with Pinocchio !!!

Hum, very interesting. I was not aware of such feature. I will have a look at it then.

That was just an example, basically it affects any rigid body algorithm. And regarding 1us for 40dof system, it is still very slow compared to Nvidia Isaac Gym, which has a simulation-to-real time ratio above 5e4 (1e-7s to perform a single integration step, requiring 5 evaluations of the dynamics since it uses RK45 internally).

@duburcqa you are a bit unfair. How many (talented) engineers are working behind Nvidia simulator?
On Pinocchio, we don't have such a task force ... in order to take profit from the latest GPU cards and so on.
We are making our best, with the money and time we have.
Concerning their simulator, I guess they are not working with generalized coordinates.
Any contribution from you @duburcqa is really appreciated to speed up and robustify Pinocchio.
By the way, if you want to speed your computation, you can leverage floating-point arithmetic computations.

I'm not criticizing Pinocchio. I think it is an amazing library, and I'm clearly willing to invest time and eventually money in it in the future if what I have in mind goes as intended. My point is, maybe there is room for speed improvement for "massively" parallel simulations taking advantage for vectorization for batch simulation or GPU thread, even for already very effective algorithms.

Thanks for the clarification.
For the moment, the initiative is to do code-generation on GPUs, with the benefit of exploiting the thread capacities of GPUs. This will be much more simple, than trying to cast RBD algorithms for batch computation according to me. But this my only answer to exploit the capacities of GPUs.

Thank you, this is insightful. I will try to look in this direction and see if I can come up with something in the next months. For now speed is not really a limitation since I have access to enough computational resources.

That was just an example, basically it affects any rigid body algorithm. And regarding 1us for 40dof system, it is still very slow compared to Nvidia Isaac Gym, which has a simulation-to-real time ratio above 5e4 (1e-7s to perform a single integration step, requiring 5 evaluations of the dynamics since it uses RK45 internally) for a system with 24 dofs on a single GPU.

Thank you for the interesting feedback. Could you give us a reference on this particular point ? I had a look to the Isaac Gym SDK and what I found was not as precise as what you wrote. And it looks like Isaac Gym is providing many many features.

If I am understanding correctly the documentation, the physical engine of Isaac Gym is either PhysX or the physical engine of Unity/fastsim/Omniverse. Then pinocchio is more comparable to PhysX.

Thank you for the interesting feedback. Could you give us a reference on this particular point ? I had a look to the Isaac Gym SDK and what I found was not as precise as what you wrote. And it looks like Isaac Gym is providing many many features.

Yes indeed it is not mentioned explicitly. It tried to infer it by myself, based on the comparison for "Learning Dexterous In-Hand Manipulation": It says that the whole learning process took 10 hours using Nvidia Isaac Gym using Nvidia A100 GPU only, while it tooks 30 hours to OpenAI to do the same. So it is 3 times faster. Then, in the above--mentioned article, it states that "This setup allows us to generate about 2 years of simulated experience per hour.", corresponding to a simulation-to-real time ratio of 17500. Next, it is mentioned that the integration step is 8ms, so computing a single integration step takes them 4.5e-7s if I'm not mistaken. Considering that Isaac is 3 times faster, it takes about 1.5e-7s to perform a single integration step. Finally, according to the documentation of Mujoco, they are using RK4 (not 45 apparently as I mentioned).

If I am understanding correctly the documentation, the physical engine of Isaac Gym is either PhysX or the physical engine of Unity/fastsim/Omniverse. Then pinocchio is more comparable to PhysX.

I don't know much about this much.

Thanks for your clarification. But without more details from NVidia on their setup and with reproducible code I found difficult to make any meaningful comparison. It is not clear that they used the same integration step than the one from the mentionned paper. They may also have used a different formulation for the simulation.

I agree with you that the information provided by Nvidia are oriented to increase the hype around their framework, to really to be informative. I guess actual benchmark will be available in the next few mouths. Yet, even though the comparison is not meaningful because of the lack of information, the overall performance (1 GPU vs 6000 CPU cores) is quite impressive.

The same goes with MuJoCo. It is a very impressive DDP for rigid body mechanical system using a specific convexification of the contact model.

Yes I very aware of their famous convexification procedure and how difficult it can be to make it works in practice. I'm quite impress by MuJoCo on this point. If you are interested in yet another contact model implementation using impulse LCP, you could have a look to tiny-differentiable-simulator by Erwin Coumans.

If you are interested by the "functional level" provided by MuJoCo then you can have a look to Crocoddyl (https://github.com/loco-3d/crocoddyl).

Yes I know Crocoddyl too but I'm not using it. I'm more interested in simulation libraries than optimal control libraries.

I agree with you that the information provided by Nvidia are oriented to increase the hype around their framework, not really to be informative unfortunately. For now it is mainly advertising. I guess actual benchmark will be available in the next few mouths. Yet, even though the comparison is not meaningful because of the lack of information, the overall performance is quite impressive (1 GPU vs 6000 CPU cores).
Yes this is quite impressive, I wonder if they used a new kind of chipset or if this is purely computer programming.

I wonder if they used a new kind of chipset

They are using their new professional flagship A100 worth 25000€ by unit. It is quite a huge technological step regarding several aspects (2To/s bandwith, GPU partioning support...). It is expected to be about 5 times more powerful than the previous flagship V100 in average for 80go variant (from 3x to 7x depending on the application).

Considering that Isaac is 3 times faster, it takes about 1.5e-7s to perform a single integration step. Finally, according to the documentation of Mujoco, they are using RK4 (not 45 apparently as I mentioned).

The numbers seem impressive. Yet, we have to make the distinction between average computation, and the call to a single simulation step. Simulation algorithms are very hard to parallelize: you can simulate several instances on different threads, but you cannot make a single step parallel due to the physics equations.
So, if you have the timings for this simple call, I would be interested and it can be used as a golden target to reach.

@jcarpent thanks for the remarks on how to exploit GPU for Pinocchio! I'm playing with GPU multi-thread programs, and I have some questions:

Does CppADCodeGen support code generation needed by Pinocchio on GPU? If not, what framework do you have in mind?
Would you elaborate on the difficulty of implementing GPU dispatcher for Pinocchio algorithms?

Thanks!

@hzyjerry Feel free to contact me on my email address to discuss this point.

@jcarpent What not to open a discussion on that topic instead ? GPU acceleration is a hop topic and I guess others may be interested to discuss this point.

@jcarpent What not to open a discussion on that topic instead ? GPU acceleration is a hot topic and I guess others may be interested to discuss this point.

Indeed, it could be beneficial for derived projects such as Crocoddyl

I don't want to animate a chatroom as I've currently a few amount of my times for that.
I would really prefer to work with people that can dedicate time soon to make a big push. @hzyjerry appears as a motivated person. If you think @duburcqa you may dedicate some of your time, I'm also open to add you to a dedicated channel.

@jcarpent No problem, I can understand that. Probably later then!


Automate the fee calculation

We need a script that pulls the needed data ( costs of x days in the past for fill, claim, and withdraw with the info of which agent is connected with the costs) for each chain and calculates the average and median costs.

This item requires some discussion.

Where we fetch the data from (Dune? / Spreadsheet?)
how much are we automizing (cronjob on a server, manual running of script?)
where do we store the fees from the last update etc.

zkSync doesn't have an etherscan block explorer so our solution needs to support multiple data sources.

create issue regarding admin roles (currently owner of the contracts updates the params)

Updated the issue name & description. Overarching issue that is pointing to all the issues related to this topic can be found here: https://github.com/beamer-bridge/beamer/issues/1600


The 'Cancel' button shows as blank on 'Create New Directory' dialog for blob container

Storage Explorer Version: 1.9.0
Build: 20190813.12
Platform/OS: Linux Ubuntu 19.04/Windows 10/macOS High Sierra
Architecture: ia32/x64
Regression From: Previous release(1.9.0)
Steps to reproduce:

Expand a Non-ADLS Gen2 account -> Blob Containers.
Create a new blob container then click 'New Folder' on toolbar.
Check the popped dialog.

Expect Experience:
The 'Create New Directory' dialog shows well.

Actual Experience:
The 'Cancel' button shows as blank on the dialog.

More Info:

This issue also reproduces for one file share.
This issue doesn't reproduce on 'New Folder' for one ADLS Gen2 blob container.
This issue also reproduces on 'Rename Directory'/'Upload' dialog.
This issue also reproduces on 'Add Entity' dialog for one table.
This issue also reproduces on 'Generate Shared Access Signature' / 'Manage Access Policies' dailog.

Updated the title and repro steps.

Closed it and reported another bug to track.


Implement IsTrajectory{Finished,Frozen} in cloud mapping

@wally-the-cartographer merge

Merge requested by authorized user cschuet. Merge queue now has a length of 1.


K.3

OS
Your OS:
Versions
The db-sync version (eg cardano-db-sync --version):
PostgreSQL version:
Build/Install Method
The method you use to build or install cardano-db-sync:
Run method
The method you used to run cardano-db-sync (eg Nix/Docker/systemd/none):
Additional context
Add any other context about the problem here.
Problem Report
Please do not include screenshots or images, but instead cut and paste any relevant log messages
or errors.

There is zero information here.  Closing this.