Decisions around summary stats ingestion

We have just realised that the rules to ingest summary stats in the pipeline has not been implemented. The README of the sumstats ingestion says:
# Edit metadata table manually (!) to
# 1. Flag which studies should be imported in the "to_ingest" column (currently, only studies that are European)
# 2. Check total sample size, case count for case-control studies
# 3. Save as configs/gwas_metadata_curated.latest.tsv

To decrease manual intervention needed for operating the release pipeline, the classification of studies needs to be fully automated. However the rules are not yet solidified. We need decisions in the following points:

[ ] What ancestries are we accepting for ingestion? Do we have exclude or include list?
[ ] What about studies with mixed  ancestries? What if 90% of the samples are European, are we still dropping the study?
[ ] What is the min case count? Min control count? What about sample sizes of continous studies? What about seriously imbalanced studies, where the case hundreds times smaller than control? Do we still ingest?
[ ] For studies that don't qualify for finemapping due to foreign ancestires, the summary stats should or shouldn't be ingested? Even top loci are not identified from the summary stats, it still can be used as a source for the PheWAS plot.

@MayaGhoussaini , what are your views in these questions?

Commenting in case it's helpful (I described some of the above steps for manually checking study ancestry)...

My heuristic was that if more than 5% of the study sample was non-EUR then I manually excluded the study from sumstat ingestion. It's not completely trivial to define the ancestry composition because the info from GWAS catalog on this is just a text field - e.g. it might say "170,911 European ancestry individuals", or it might say "99 Italian ancestry cases, 359 Italian ancestry smoker controls". A few rules could capture most cases, but there will probably be examples that fail to map in an automated pipeline. In addition,  I think that applying fine-mapping as we do now with a UKB ref panel for general EUR samples is questionable, and my impression is that the field is moving away from this - i.e. to say that fine-mapping with out-of-sample LD is dangerous (will give you false associations).

We should probably have a minimum total sample size limit, but it's hard to know what it should be. An alternative would be to compute a genomic inflation factor and exclude studies based on that, which would be quite straightforward. This could automatically exclude some problematic studies, such as those with highly imbalanced case/control ratio which don't adjust for it properly. But if the study has adjusted for case/control imbalance, then there's no reason in principle why it should be excluded.

This is a good point. Our pipelines currently take all sumstats that are ingested to run fine-mapping on. I.e. we don't distinguish sumstat studies that should only go into PheWAS from those that should have fine-mapping/coloc. It would be good to ingest sumstats from studies with different ancestries, but not to run fine-mapping on those.

Hi @DSuveges, I can see that Jeremy already responded to the questions and I agree with all his reasoning. Thanks Jeremy! This could only change if we suddenly get hold of non-European reference panels that will enable us to perform downstream work on GWAS (i.e. FM).  Giving that you raised comment number 3, I think we should aim to ingest all sumstats (including mixed ancestries&non-Europeans). These will go into PheWAS but not down the fine-mapping/coloc path. Happy to discuss this more in person if needed

Another important note that is not directly related but would be good to have it in the new layout of the portal is to have the curated studies from GWAS Cat also feeding into PheWAS.

Hi @Jeremy37 , Thank for the insights! This is indeed a hard nut to crack.
I was planning to use the downloadable ancestry file from the GWAS Catalog, that has a more structured way to annotate ancestries (when available form publication to curate). The data looks like this:
+---------------+-------+--------------------+-----------------------------------------------------------------+---------------------------------------------------------------------------+
|STUDY ACCESSION|STAGE  |NUMBER OF INDIVDUALS|BROAD ANCESTRAL CATEGORY                                         |COUNTRY OF RECRUITMENT                                                     |
+---------------+-------+--------------------+-----------------------------------------------------------------+---------------------------------------------------------------------------+
|GCST90017131   |initial|14306               |European                                                         |Canada, Netherlands, Sweden, U.S., Belgium, Finland, Denmark, U.K., Germany|
|GCST90017131   |initial|811                 |East Asian                                                       |Republic of Korea                                                          |
|GCST90017131   |initial|1097                |Hispanic or Latin American                                       |U.S.                                                                       |
|GCST90017131   |initial|114                 |African American or Afro-Caribbean                               |U.S.                                                                       |
|GCST90017131   |initial|481                 |Greater Middle Eastern (Middle Eastern, North African or Persian)|Israel                                                                     |
|GCST90017131   |initial|1531                |NR, Other admixed ancestry                                       |Canada, Netherlands                                                        |
+---------------+-------+--------------------+-----------------------------------------------------------------+---------------------------------------------------------------------------+

While the initial sample description of the same study looks like this:
14,306 European ancestry individuals, 811 Korean ancestry individuals, 1,097 Hispanic individuals, 114 African American individuals, 481 Middle Eastern ancestry individuals, 1,531 unknown and other admixed ancestry individuals`

So curators, irrespectively the country of recruitment, assign the broader ancestry category. This value is always filled. The caveat is that these categories are only split if the authors provide granular description and sample sizes. Unless such cases can happen:
-RECORD 0----------------------------------------------------------------------------------------------------------
 STUDY ACCESSION                | GCST003500
 PUBMEDID                       | 27114598
 FIRST AUTHOR                   | Liu C
 DATE                           | 2016-04-25
 INITIAL SAMPLE DESCRIPTION     | 71 cases with pancreatitis, 142 cases without pancreatitis
 REPLICATION SAMPLE DESCRIPTION | NA
 STAGE                          | initial
 NUMBER OF INDIVDUALS           | 213
 BROAD ANCESTRAL CATEGORY       | NR, European, Hispanic or Latin American, African unspecified, Asian unspecified
 COUNTRY OF ORIGIN              | NR
 COUNTRY OF RECRUITMENT         | U.S.
 ADDITONAL ANCESTRY DESCRIPTION | null

If we treat the BROAD ANCESTRAL CATEGORY as a comma separated list of ancestries the situation is manageable.

So the plan is:

Upon processing study level metadata, capturing sample descriptions, case/control numbers as granular as possible to enable downstream decision making at the relevant step of the pipeline.
This metadata can tell to drop sumstats based on low sample sizes or other parameters (parameter defined in the config).
Otherwise, if there's a harmonized summary statistics dataset on the gwas catalog ftp, we'll try to ingest it. No exclusion based on ancestry at this stage.
However some filters can also applied upon ingestion, that renders a summary statistics un-eligible: if certain fields are missing (eg. effect). Also we can decide to implement some quality metrics to detect uncontrolled inflation in the data.
Upon ingestion some of the missing values eg. SE can be computed based p-value and effect size. (it's already being done in v2d)
Downstream to the summary statistics ingestion, the finemapping step should be responsible to make decision on which studies should be fine mapped. The logic should take into account the ancestry, required statistical fields etc. These criteria will be defined later.

@DSuveges could you store this info somewhere else and close the ticket? I think the discussion phase is over

We can consider this effort closed, implementation is covered under #2892.


Client Missing Received Messages

Describe the bug
It appears we are sometimes missing messages in high volume. Using the ManagedMqttClient I'm subscribing to a topic, sending a message to that topic, reading that message back from the subscription (or in the cases where we miss messages I'm waiting 10 seconds and then continuing) and then unsubscribing. I'm doing this in a loop for 200 messages and every time it misses at least 1 message, sometimes even more. When I switch over to the regular MqttClient, this does not occur. I've verified that this happens (maybe even worse) in release mode. I've verified that the messages are actually coming through using mosquitto_sub but the ApplicationMessageReceivedAsync is just not firing off using ManagedMqttClient for these few messages. I am also using the latest version, <IP_ADDRESS>3.
Which component is your bug related to?

ManagedClient

To Reproduce
Steps to reproduce the behavior:

Have a broker setup
Update the code attached to use your broker, topic, and message
Run the code
Check the number of messages received at the end, if it is not 200 then a message was missed

Expected behavior
All 200 response messages should be seen and handled
Screenshots

Code example
MqttNetMissingMessagesExample.zip
using MQTTnet;
using MQTTnet.Client;
using MQTTnet.Extensions.ManagedClient;
using MQTTnet.Protocol;
using System.Collections.Concurrent;

internal class Program
{
    private static readonly ConcurrentDictionary<string, IEnumerable<(string message, int sequence, DateTime timeOfMessage)>> _DeviceMessageDict = new();

    private static async Task<string> WaitForMessage(string topic, DateTime earliestMessageTime, TimeSpan timeout, TimeSpan timeBetweenChecks)
    {
        if (string.IsNullOrWhiteSpace(topic))
        {
            throw new ArgumentNullException(nameof(topic));
        }

        var startTime = DateTime.UtcNow;
        while (DateTime.UtcNow - startTime < timeout)
        {
            if (_DeviceMessageDict.TryGetValue(topic, out var messages))
            {
                var messageInfo =
                    messages
                    .Where(messageInfo => messageInfo.timeOfMessage > earliestMessageTime)
                    .OrderByDescending(messageInfo => messageInfo.sequence)
                    .Select(messageInfo => new { Message = messageInfo.message })
                    .FirstOrDefault();

                if (messageInfo is not null)
                {
                    var message = messageInfo.Message;
                    _DeviceMessageDict.TryUpdate(topic, messages.Where(m => m.message != message).ToList(), messages);
                    return message;
                }
            }

            await Task.Delay(timeBetweenChecks).ConfigureAwait(false);
        }
        return $"No message received from {topic} after timeout of {timeout}";
    }

    private static async Task Main()
    {
        var mqttFactory = new MqttFactory();

        using (var mqttClient = mqttFactory.CreateManagedMqttClient())
        {
            var mqttClientOptions = new MqttClientOptionsBuilder()
                .WithTcpServer("YOUR TCP SERVER")
                .WithWillQualityOfServiceLevel(MqttQualityOfServiceLevel.AtMostOnce)
                .WithWillRetain(false)
                .WithCredentials("YOUR USERNAME", "YOUR PASSWORD")
                .Build();

            var managedMqttClientOptions = new ManagedMqttClientOptionsBuilder()
                .WithAutoReconnectDelay(TimeSpan.FromSeconds(5))
                .WithClientOptions(mqttClientOptions)
                .Build();

            await mqttClient.StartAsync(managedMqttClientOptions);

            int counter = 0;
            mqttClient.ApplicationMessageReceivedAsync += e =>
            {
                var count = Interlocked.Increment(ref counter);
                var message = e.ApplicationMessage.ConvertPayloadToString();
                var topic = e.ApplicationMessage.Topic;

                Console.WriteLine($"Received {count} message {message} on Topic {topic}");

                _DeviceMessageDict.AddOrUpdate(
                    topic,
                    new List<(string, int, DateTime)>() { (message, 1, DateTime.UtcNow) },
                    (k, v) => v.Append((message, v.Count() + 1, DateTime.UtcNow))
                    );

                return Task.CompletedTask;
            };

            for (int i = 0; i < 200; i++)
            {
                var topic = "YOUR TOPIC";
                var filter = new[]
                {
                    new MqttTopicFilterBuilder().WithTopic(topic).Build()
                };
                await mqttClient.SubscribeAsync(filter).ConfigureAwait(false);

                var startTime = DateTime.Now;
                var message = "YOUR MESSAGE";
                await mqttClient.EnqueueAsync(topic, message);
                Console.WriteLine($"{message} sent to topic {topic}");

                var result = await WaitForMessage(topic, startTime, TimeSpan.FromSeconds(10), TimeSpan.FromMilliseconds(100));
                Console.WriteLine($"Result: {result}");
                await mqttClient.UnsubscribeAsync(topic).ConfigureAwait(false);
            }

            Console.WriteLine($"Total messages: {counter}");
        }
    }
}

Between this and 1849 the managed client is completely unusable. I am going to have to use the regular client as a work around, it does seem to be behaving as it should. But these items really need fixed


Automatically support HEAD method for all GET routes, as Starlette does

First check

[x] I added a very descriptive title to this issue.
[x] I used the GitHub search to find a similar issue and didn't find it.
[x] I searched the FastAPI documentation, with the integrated search.
[x] I already searched in Google "How to X in FastAPI" and didn't find any information.
[x] I already read and followed all the tutorial in the docs and didn't find an answer.
[x] I already checked if it is not related to FastAPI but to Pydantic.
[x] I already checked if it is not related to FastAPI but to Swagger UI.
[x] I already checked if it is not related to FastAPI but to ReDoc.
[x] After submitting this, I commit to:

Read open issues with questions until I find 2 issues where I can help someone and add a comment to help there.
Or, I already hit the "watch" button in this repository to receive notifications and I commit to help at least 2 people that ask questions in the future.
Implement a Pull Request for a confirmed bug.

Example
Here's a self-contained minimal, reproducible, example with my use case:
from fastapi import FastAPI
import pytest
from fastapi.testclient import TestClient

app = FastAPI()

@app.get("/")
def read_root():
    return {"Hello": "World"}

client = TestClient(app)

def test_index_head():
    response = client.head("/")
    assert response.status_code == 200

Description

The above test fails because the HEAD request returns a 405.
This is counterintuitive.

The solution you would like

To better support the HTTP standard, all routes that handle GET methods should automatically handle HEAD methods, too.
This is similar to what Starlette's router already does: https://github.com/encode/starlette/issues/45
This should happen without requiring the developer to do any additional work.

Describe alternatives you've considered

It's currently possible to do this manually by adding @app.head and a helper method for each route, but it's cumbersome:

@app.head("/")
def read_root_head():
    return Response()

It could also be done using middleware, but that would incur a performance overhead.

Environment

OS: [e.g. Linux / Windows / macOS]: Linux
FastAPI Version [e.g. 0.3.0]: 0.60.1
Python version: 3.8.5

encode/starlette#45 is only about static FileResponse. I don't know how a default should be set for a dynamic API.

Not so, if you look at the related commit at https://github.com/encode/starlette/pull/132/files in starlette/routing.py, you can see Starlette adds HEAD methods automatically for every Route that supports GET, regardless of whether they are FileResponse or not.
Many web frameworks do this by default too, e.g. Flask (https://flask.palletsprojects.com/en/1.1.x/quickstart/#http-methods) and Django. It makes sense to do that because a server should respond to an HTTP HEAD request as defined in the spec, see e.g.:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/HEAD
"The HTTP HEAD method requests the headers that are returned if the specified resource would be requested with an HTTP GET method. Such a request can be done before deciding to download a large resource to save bandwidth, for example."
Clients use this for cache invalidation. The server just needs to run the route method, respond with the headers but discard the response body. Ideally, FastAPI would automatically add an OPTIONS method handler, too. This would help make FastAPI a first class web framework.
This is safe even for a dynamic API, because to follow REST principles, HTTP GET routes should always be idempotent (as should PUT, DELETE, HEAD, OPTIONS and TRACE). Only HTTP PUTs are allowed not to be idempotent.
This behavior could be controlled by a flag defaulting to true.

working on this

Thanks @matthewlloyd for the very clear argument and especially for the self-contained example, that helps a lot.
Indeed, here in the spec: https://tools.ietf.org/html/rfc7231#section-4.3.2 it says:

The server SHOULD send the same header fields in response to a HEAD request as it would have sent if the request had been a GET [...]

And the OpenAPI spec doesn't say anything contradicting it, so FastAPI should support it.
I think ideally, it should handle HEAD requests automatically for GET requests, without adding them to the OpenAPI schema. I think HEAD operations should be added to OpenAPI only when explicitly declared in the code (ideally).

Thanks @victorphoenix3 for working on this! Let me know if you need any help with it.

@victorphoenix3 Any progress on this issue?  This bit me today.  Is it just a matter of pulling from upstream?

I'm on it.

Implemented second approach. Needs a refactor, and I need to think if there's incompatibility with route.Route from Starlette.
I'll be waiting for @tiangolo's instructions before implementing/refactoring what is missing.

@Kludex with either approach, we'll need to return the same headers with get/head to be spec-compliant (as @tiangolo has already pointed out) so a lot of test logic with GET can be re-used sans body

This bit me today when implementing a Docker healthcheck using wget --spider (which sends a HEAD request). Any updates on this?

Got the same issue as our app is monitored by uptimerobot

Bumping this thread, we are using this for health-checks, and pre-fetch verifications. Any updates would be great :eyes:

Btw if it's possible to help, I'm more than willing to.

i found this issue today, and i play a bit, and this is what works for me (FastAPI 0.70.0):
@router.head("/hello")
@router.get("/hello")
def read_root():
    return {"Hello": "World"}

so with httpie this are the results. for GET method
$ http http://localhost:8080/hello

HTTP/1.1 200 OK
content-length: 17
content-type: application/json
date: Mon, 06 Dec 2021 15:14:18 GMT
server: uvicorn

{
    "Hello": "World"
}

and then for HEAD method:
$ http head http://localhost:8080/hello

HTTP/1.1 200 OK
content-length: 17
content-type: application/json
date: Mon, 06 Dec 2021 15:14:15 GMT
server: uvicorn

To avoid duplicating the request line, @router.api_route can be used:
@router.api_route(url, methods=['GET', 'HEAD'])

Hey. I was taking a look at open issues I could contribute to and I stumble with this one.
A way to implement this behavior without changing Fastapi or remembering to add the head decorators is to define an startup hook that add those head endpoints for you:
  @app.on_event("startup")
  def add_default_head_endpoints(self) -> None:
        for route in self.routes:
            if isinstance(route, routing.APIRoute) and "GET" in route.methods:
                new_route = copy(route)
                new_route.methods = {"HEAD"}
                new_route.include_in_schema = False
                self.routes.append(new_route)

As this is a desired behavior for fastapi, I created a PR that implements this hook as a builtin

Hey. I was taking a look at open issues I could contribute to and I stumble with this one.
A way to implement this behavior without changing Fastapi or remembering to add the head decorators is to define an startup hook that add those head endpoints for you:
  @app.on_event("startup")
  def add_default_head_endpoints(self) -> None:
        for route in self.routes:
            if isinstance(route, routing.APIRoute) and "GET" in route.methods:
                new_route = copy(route)
                new_route.methods = {"HEAD"}
                new_route.include_in_schema = False
                self.routes.append(new_route)

As this is a desired behavior for Fastapi, I created a PR that implements this hook as a builtin. Feel free to take a look at it 😄

It's not desired to include routes on startup. I have two PRs about it already. I'll check later which one should be kept to review.

Hey. I was taking a look at open issues I could contribute to and I stumble with this one.
A way to implement this behavior without changing Fastapi or remembering to add the head decorators is to define an startup hook that add those head endpoints for you:
  @app.on_event("startup")
  def add_default_head_endpoints(self) -> None:
        for route in self.routes:
            if isinstance(route, routing.APIRoute) and "GET" in route.methods:
                new_route = copy(route)
                new_route.methods = {"HEAD"}
                new_route.include_in_schema = False
                self.routes.append(new_route)

As this is a desired behavior for Fastapi, I created a PR that implements this hook as a builtin. Feel free to take a look at it 😄

It's not desired to include routes on the "startup" event. I have two PRs about it already. I'll check later which one should be kept to review.

I did take a look at your PRs and, as they were inactive, I decided to propose another approach. However, if you are going to retake the issue you'll probably know better than me the proper design decisions to take 😁

Well... The PRs are not inactive... They were just not reviewed. 😅
But I improved my skills since I developed them, so I'll take another view.

and this is biting me as well :(
HAproxy sends a HEAD request every few seconds to make sure the site is up
i used @juantoca 's startup since there is no ways im going through a f ton of endpoints to add the head method to them
a slight change since im not using a class for the app part
from fastapi.routing import APIRoute

@app.on_event("startup")
async def add_default_head_endpoints() -> None:
    for route in app.routes:
        if isinstance(route, APIRoute) and "GET" in route.methods:
            new_route = copy(route)
            new_route.methods = {"HEAD"}
            new_route.include_in_schema = False
            app.routes.append(new_route)

INFO:     xxx.xxx.xxx.xxx:42028 - "HEAD / HTTP/1.1" 200 OK
2023-11-13 09:32:26 1790 DEBUG:fastapi_events.middleware: Processing events
2023-11-13 09:32:26 1790 DEBUG:fastapi_events.middleware: Resetting event_store ctx
2023-11-13 09:32:41 1790 DEBUG:fastapi_events.middleware: Setting event_store ctx

im on fastapi-0.104.1 and python 3.12

2024 is going to end, and it's not looking good.
I came across this while setting up uptime monitoring.


Issues w/ tutorial NEQ run

I've tried running the NEQ example as outline in the documentation:

from openmm import unit

from absolv.models import (
    NonEquilibriumPfrom openff.toolkit.typing.engines.smirnoff import ForceField
force_field = ForceField("openff-2.0.0.offxml")

from absolv.runners.equilibrium import NonEquilibriumRunner
NonEquilibriumRunner.setup(schema, force_field)rotocol,
    SwitchingProtocol,
    SimulationProtocol,
    State,
    System,
    TransferFreeEnergySchema
)
from absolv.runners.nonequilibrium import NonEquilibriumRunner
from openff.toolkit.typing.engines.smirnoff import ForceField

schema = TransferFreeEnergySchema(
    system=System(solutes={"CCO": 1}, solvent_a=None, solvent_b={"O": 895}),
    # Define the state that the calculation will be performed at.
    state=State(temperature=298.15 * unit.kelvin, pressure=1.0 * unit.atmosphere),
    # Define the alchemical pathway to transform the solute along in vacuum ('solvent_a')
    # and water ('solvent_b')
    alchemical_protocol_a=NonEquilibriumProtocol(
        production_protocol=SimulationProtocol(
            n_steps_per_iteration=6250,
            n_iterations=160,
            timestep=2.0 * unit.femtoseconds
        ),
        switching_protocol=SwitchingProtocol(
            n_electrostatic_steps=60,
            n_steps_per_electrostatic_step=100,
            n_steric_steps=0,
            n_steps_per_steric_step=0,
            timestep=2.0*unit.femtoseconds
        )
    ),
    alchemical_protocol_b=NonEquilibriumProtocol(
        production_protocol=SimulationProtocol(
            n_steps_per_iteration=6250,
            n_iterations=160,
            timestep=2.0 * unit.femtoseconds
        ),
        switching_protocol=SwitchingProtocol(
            # Annihilate the electrostatic interactions over the first 12 ps
            n_electrostatic_steps=60,
            n_steps_per_electrostatic_step=100,
            # followed by decoupling the vdW interactions over the next 38 ps
            n_steric_steps=190,
            n_steps_per_steric_step=100,
        )
    )
)

force_field = ForceField("openff-2.0.0.offxml")
NonEquilibriumRunner.setup(schema, force_field)

However, the above gives the error:
Traceback (most recent call last):
  File "testRun.py", line 52, in <module>
    NonEquilibriumRunner.setup(schema, force_field)
  File "/home/spadavec/itx/research_projects/absolv/absolv/runners/_runners.py", line 171, in setup
    cls._setup_solvent(
  File "/home/spadavec/itx/research_projects/absolv/absolv/runners/_runners.py", line 96, in _setup_solvent
    topology, coordinates = PACKMOLCoordinateFactory.generate(components)
  File "/home/spadavec/itx/research_projects/absolv/absolv/factories/coordinate.py", line 147, in generate
    box_size = cls._approximate_box_size_by_density(components, target_density)
  File "/home/spadavec/itx/research_projects/absolv/absolv/factories/coordinate.py", line 68, in _approximate_box_size_by_density
    volume += molecule_volume * count
  File "/home/spadavec/miniconda3/envs/absolv2/lib/python3.8/site-packages/openmm/unit/quantity.py", line 235, in __add__
    raise TypeError('Cannot add two quantities with incompatible units "%s" and "%s".' % (self.unit, other.unit))
TypeError: Cannot add two quantities with incompatible units "angstrom**3" and "mole*milliliter/gram".

Is there something I need to change? I should also note in the documentation, the NEQ example needs to change the following line(s):
from
from absolv.runners.equilibrium import NonEquilibriumRunner

to
from absolv.runners.nonequilibrium import NonEquilibriumRunner

and SwitchingProtocol is never imported from absolv.models
Cheers

Thanks for reporting this, and sorry it took so long to see it!
I've spent some time re-writing the framework (#67), and this issue should hopefully be fixed in the new version that will be released shortly.


enable sphinx.ext.napoleon

Enables the napoleon extension in sphinx. This will interpret the numpydoc style parameters and types and convert them to sphinx :param type name: Note that sphinx.ext.napoleon must come before the numpydoc extension.
Eventually the numpydoc dependency might be able to be removed but currently removing it makes the wrapped ufunc documentation omit the "parameters" and "returns", see the attached screenshot for what one of these ufuncs looks like with numpydoc removed.

[ ] Closes #3056

Thanks @DocOtak !
It's difficult to see the results without merging - are you comfortable enough with this that we can merge and assess the result? Or would you like someone to build the docs locally and check first?

I was diffing the directory outputs when testing locally and nothing really breaking stood out to me... I think this is safe enough that it should be able to merge and asses the results. IMO the results were quite pretty and definitely addressed #3056 in most places.

Thanks @DocOtak


getting started

Is this ready to use and is there an example?

If you want to just try out the libraries, if you create an account here: www.hops.site, i can activate the account. Then follow the "TensorFlow tour" - there are sample hyperparameter opt notebooks (fashion_mnist), TensorFlowOnSpark, and Horovod.
If you want to install hopsworks, try it out with a vm first using:
git clone https://github.com/hopshadoop/karamel-chef.git
cd karamel-chef
./run.sh ubuntu 1 hopsworks no-random-ports

Documentation is now available here.


Feature: training dataset maintenance

Context
Now that we've done it a few times, let's be systematic about how we update the base training dataset in Hugging Face
Requirements

[ ] establish training-urls as the canonical dataset in Hugging Face (HF), which we will use Pull Requests in to maintain. We will likely remove the other ones.
[ ] add a last-updated property

We then need a script which we can trigger with GitHub Actions every few weeks to update our training data.

[ ] check for new URLs in our database which aren't already in the training-urls dataset via the API
[ ] run the HTML tag collector on all URLs missing all tag collector properties
[ ] check for new URLs from label studio (requires #19)
[ ] update the last-updated property
[ ] use the HF API to open a Pull Request for the training-urls dataset
[ ] set up a GitHub Action to trigger this manually

Docs

update this repo's readme with reference to the script and action

[ ]  check for new URLs in our database which aren't already in the training-urls dataset via the API

Where will these new raw URLs be hosted? At the moment, my common_crawler pr #45 simply stores new URLs in the repository, which obviously isn't a sustainable long-term option.

@maxachis this is a good point—in general, I think we should use hugging face datasets.
I added detail to this issue: https://github.com/Police-Data-Accessibility-Project/data-source-identification/issues/40

@josh-chamberlain To make sure I fully understand the workflow:

Pull URLs from database
Pull URLs from training-urls dataset
Get all URLs from 1 which are not in 2
Run HTML tag collector on results from 3
Insert these results into LabelStudio (need confirmation especially on this step)
Take results from LabelStudio
Merge with URLs from training-urls dataset pulled in 2. Update the last_updated property of all new entries (or of the entire dataset?)
Put the results of 7 into training-urls dataset

Additionally, when we are talking about training-urls dataset, does this dataset currently exist? In PDAP's Hugging Face, I do not currently see a dataset named training-urls:

@maxachis

we don't need to insert into LabelStudio—to be clear, we are checking LabelStudio for newly labeled URLs which aren't already in our training data.

training-urls doesn't currently exist, creating the dataset + strategy for managing batches of URLs within it.

@maxachis I updated the readme for this repo and tweaked this issue slightly—I think using Hugging Face as a database for un-labeled URLs is not needed. We can track batches by ID in github, but we don't need to put them in hugging face before they're labeled. Hopefully this is much simpler.


chore(deps) bump tabwriter, windows-core, tempfile, memchr, clang-sys

This bundles up 5 dependabot PRs and updates the related dependencies.
ref : #2603 , #2604 , #2605 , #2606 , #2607

Codecov Report

Merging #2608 (be158ab) into main (6f1db26) will increase coverage by 0.01%.
Report is 7 commits behind head on main.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2608      +/-   ##
==========================================
+ Coverage   65.88%   65.89%   +0.01%
==========================================
  Files         133      133
  Lines       16819    16819
==========================================
+ Hits        11081    11083       +2
+ Misses       5738     5736       -2


ImportError: cannot import name 'APIZenodo' from 'zen.api'

Hi @rolfsimoes !
Just started having a look at zen as I am currently looking for a lib to interact with Zenodo.
On cell 2 of https://github.com/Open-Earth-Monitor/zen/blob/main/examples/basics.ipynb
When I run from zen.api import APIZenodo
I get
ImportError: cannot import name 'APIZenodo' from 'zen.api'
Is that expected ?

Hi @oolonek !
Yep. This file was written in a early version of zen and should be updated. The class APIZenodo is now an internal class, so I changed its name to _APIZenodo. To interact with Zenodo API you can use the Zenodo class. You can see some examples in the documentation. However, I didn't implemented the license and records endpoints shown in the notebook you are trying to run. At the end of this week I will release some updates to fix this.


Trigger

Hi,
does this driver support for setting up the external trigger function of the camera? I'd like to use it in Matlab, but need the ability to trigger the camera.
Thanks!

I have not used external triggers so far. You'd have to try that manually, by configuring the camera to use an external trigger with the baslerSetParameter function and reading the images with baslerGetData. I guess that should work, but I don't have any experience with the external trigger function. Sorry I can't give you a more detailed answer.

Thanks, i will give it a try and report back!

Hi,
I'm a colleague of ebauch.
Recently I have been trying to set up a Basler USB3.0 camera, and I encountered a problem with the TriggerMode parameter: it cannot be set as ‘On' for data acquisition. To be specific, the TriggerMode parameter can be set as ‘On’ using either the baslerSetParameter function or the Pylon software (the status of the parameter is verified in Pylon). However, once the baslerGetParameter function or the baslerGetData function or the baslerSetParameter function (setting up other parameters) is applied to the camera, this TriggerMode parameter would automatically be ‘Off’. Besides, other parameters such as ‘Width’ and ‘PixelFormat’ work fine with all the functions above.
My naive guess would be that the camera initialization overwrites some settings, but couldn't figure out why and where it is. Would you have any suggestions? Thanks a lot!

Hi,
All the functions in this driver are like this:
camera.Open();
DoSomething
camera.Close();
This is a safe approach, since you can't leave the camera open, but for the price of not able to use TriggerMode 'On'. (By the way, I found another parameter that you can't change in this way:
For AcquisitionMode the default value is 'Continuous', but in triggered mode you may want to switch to 'SingleFrame')
You should really use camera.Open(); only once, at the beginning of your code and camera.Close(); once as well at the end.
But this means you should rewrite every function here :(

Hey, I’m now at the same problem as you were a couple of years ago. Did you sort out any solution for the problem with the trigger?


ion-slides loop with dynamic content doesn't behave correctly

Ionic version:  (check one with "x")
[ ] 1.x
[X ] 2.x
[ ] 3.x
I'm submitting a ...  (check one with "x")
[ X ] bug report
[ ] feature request
[ ] support request => Please do not submit support requests here, use one of these channels: https://forum.ionicframework.com/ or http://ionicworldwide.herokuapp.com/
Current behavior:
When initialSlide is set to 1, the first slide presented is 2 (the second slide). So I guess initialSlide counts from 0, but getActiveIndex() counts from 1.
With three slides and loop="true" you can still slide to slide 0 or slide 4 and get a blank slide. Sliding again will give make the slider skip a slide:
Slide 3 -> Slide 4 (blank) -> Slide 2 (skipped 1).
The same will happen the other way.
Expected behavior:
The loop should go like 1->2->3->1->etc, OR 0->1->2->0 depending on how you count.
Steps to reproduce:
Create blank project and create three slides with images which are bound to a variable.
See plunkr: http://plnkr.co/edit/RezOKP2Rb01hMB60KbM8?p=preview
However, not binding the sources to a variable will make the slides work as they should:
http://plnkr.co/edit/r6L91UqQ3jQbSL12NDUD?p=preview
But it still shows the wrong indexes in getActiveIndex
Other information:
Cordova CLI: 6.5.0
Ionic Framework Version: 3.0.1
Ionic CLI Version: 2.2.1
Ionic App Lib Version: 2.2.0
Ionic App Scripts Version: 1.3.0
ios-deploy version: 1.9.0
ios-sim version: 5.0.10
OS: macOS Sierra
Node Version: v7.7.4
Xcode version: Xcode 8.3 Build version 8E162

Hello, thanks for using Ionic! I am going to close this issue as a duplicate of https://github.com/driftyco/ionic/issues/10784.


cleanup created bundle when creating failed

If any error occurs, we should cleanup the bundle we created. If not, this may confuse users.
Signed-off-by: Ma Shimiao<EMAIL_ADDRESS>

LGTM

LGTM


Custom Button on the Player

A custom button through DOM manipulation on player ( buttons like skip intro ) to appear when needed and destroyed when they fullfilled their purpose.

Seconded, however it might be better as an overlay that is controlled by the current video timestamp (probably in milliseconds). This seems to be a really specific feature though that is most likely outside the scope of the project. You might have better luck forking and adding the feature yourself if you desperately need it.

Not very special.  Let's say I want to add "Next" / "Previous" buttons to a video playlist.  And I want these buttons to be with other controls, or at least appear and disappear with them

is there a way to do this currently or no?

If you have a solution, please let me know. I need it too. Thank you.


update the path where we look for the provider.d file to appease rust-analyzer

@pfmooney observed that rust-analyzer runs builds from a different cwd than if one runs cargo build by hand. Consequently, the dtrace_provider! macro looks for the provider file in the wrong place. This PR uses CARGO_MANIFEST_DIR as the base directory in which we look for the specified provider file.

@bnaecker no rush, but can you publish a new version on crates.io when you have a moment?


Add onAuthorizedBeforeSubmit for ApplePay

Is your feature request related to a problem? Please describe.
Having access to the apple pay event before submitting is important. Adding an event that isonAuthorizedBeforeSubmit would help solve this so you don't have to implement it with apple pay without Adyen.
Describe the solution you'd like
Add onAuthorizedBeforeSubmit event and execute here:
onPaymentAuthorized: (resolve, reject, event) => {
    if (!!event.payment.token && !!event.payment.token.paymentData) {
        this.setState({ applePayToken: btoa(JSON.stringify(event.payment.token.paymentData)) });
    }

+   if(onAuthorizedBeforeSubmit) {
+      await new Promise((resolve) => onAuthorizedBeforeSubmit(resolve, reject, event));
+  }
    super.submit();
    onPaymentAuthorized(resolve, reject, event);
}

**Describe alternatives you've considered**
Adding the apple pay `event` to the `beforeSubmit` event.

Hey Aral, what is the use case?

we would like to create an express flow with Apple Pay that just pressing the "Pay with Apple Pay" button will work even if users are not logged in or not registered. We would like to create the order in our system before the payment is authoritzed in Adyen and to be able to create the order, we need the shipping address. We do not want an authorized payment that is not yet associated with any order / user in our system.

Hi @aralroca . Thanks for reaching out and proposing the current solution 🙌
We are already discussing internally how to improve ApplePay and other payment methods in order to make them work better for your scenario. There is already a task in our backlog to improve the ApplePay and return the shipping address - we haven't started it yet though.
As a quick workaround for now, I suggest you to use the onAuthorized callback to do the submit part: you can get the details from the event object, create the order on your end, and then do what you would do within the onSubmit block.
In this way, you would not use the onSubmit callback. It will be triggered, but you can ignore it and do the whole flow within the onAuthorized. Let me know if that works.

Hi @ribeiroguilherme. Thanks for your proposal as a temporal workaround meanwhile this process is not managed in a better way.
Unfortunately, I get "Payment not completed" in the Apple popup, this error is hard to catch:

onError is not executed
onPaymentCompleted is not executed
Any error in the console or network

const appleConfig = {
  ...restOfConfig,
  // void onSubmit
  onSubmit: () => {},
  // onAuthorized calling own submit endpoint
  onAuthorized: async (resolve, reject, event) => {
    // ... here the rest of code to save the user, etc
    const { status, data } = await submit({
      token: cart.token,
      paymentId,
      applePayToken: JSON.stringify(event.payment.token.paymentData),
     // Other fields are fulfilled by our endpoint
    })
    console.log(status) // => 200
    console.log(data) // => { "result": "Authorised" }
    if (status === 200 && data.result === 'Authorised') resolve()
    else resolve()
  },
}

Do you have any idea about what causes this issue and how to solve it? Looks that after the resolve is not detecting that the submit was done... Or is it something I miss doing?
Thank you

Hi @aralroca ,
By checking the code, it seems to me that it should work as expected when you call the resolve() method.  Resolving the promise should trigger the completePayment and then you would get the successful message.
Do you have any other findings on your side? Did you manage to get any error on the console or network tab?

Glad to hear it worked. Cheers! 👌


[mesheryctl] [child] make system logs platform aware(platform: kubernetes)

See Epic #2307 "provide support for platform: kubernetes" for context and further description of this issue.
Current Behavior

mesheryctl system logs will start logs from Meshery server running on Docker. The current implementation defaults to Docker as the platform and the logs are obtained as shown-
https://github.com/layer5io/meshery/blob/b0c1a7fa39dd0456f9112be068d2601bfe17d465/mesheryctl/internal/cli/root/system/logs.go#L51
Desired Behavior

mesheryctl system logs should check the current platform and if the platform is Kubernetes, it should get the logs from Kubernetes. (May involve port-forwarding/streaming from Kube-API)
// Adding @abhishekkumkar to this thread.

@piyushsingariya Assigning it to you 🚀

Wait, I can't seem to assign it to you.

I'm not sure. Why this is happening?

there you go

Do you need to be on the thread to be assigned? I was not able to add @piyushsingariya that is why I asked to comment on the thread.


Photos showing in landscape

Some of the photos that I choose from the picker are displaying in landscape or wrong orientation.

Sorry! This UI is not support landscape.

that is not what I am saying. I am saying that the picker is turning the
image.
On Mon, Apr 27, 2015 at 1:13 AM, wenzhaot<EMAIL_ADDRESS>wrote:

Sorry! This UI is not support landscape.
—
Reply to this email directly or view it on GitHub
https://github.com/wenzhaot/InstagramPhotoPicker/issues/11#issuecomment-96508990
.

I am experiencing the same issue @codeSupreme.
Were you able to fix it or have a workaround?
Thanks a lot

The problem is that the images you are using were taken in portrait mode with the iPhone camera. This means that their UIImageOrientation is equal to right... You need to just rotate the image.. i.e. if your image is right then you need to rotate it 90 degrees. There are a ton of stack articles on this.
Here is what i do to properly orient images up no matter their input direction:

(UIImage *)imageRotate{
if (self.imageOrientation == UIImageOrientationUp)
{
return self;
}
CGAffineTransform transform = CGAffineTransformIdentity;
switch (self.imageOrientation)
{
case UIImageOrientationDown:
case UIImageOrientationDownMirrored:
transform = CGAffineTransformTranslate(transform, self.size.width,self.size.height);
transform = CGAffineTransformRotate(transform, M_PI);
break;
  case UIImageOrientationLeft:
  case UIImageOrientationLeftMirrored:
      transform = CGAffineTransformTranslate(transform, self.size.width, 0);
      transform = CGAffineTransformRotate(transform, M_PI_2);
      break;

  case UIImageOrientationRight:
  case UIImageOrientationRightMirrored:
      transform = CGAffineTransformTranslate(transform, 0, self.size.height);
      transform = CGAffineTransformRotate(transform, -M_PI_2);
      break;
  default:
      break;

}
switch (self.imageOrientation)
{
case UIImageOrientationUpMirrored:
case UIImageOrientationDownMirrored:
transform = CGAffineTransformTranslate(transform, self.size.width, 0);
transform = CGAffineTransformScale(transform, -1, 1);
break;
  case UIImageOrientationLeftMirrored:
  case UIImageOrientationRightMirrored:
      transform = CGAffineTransformTranslate(transform, self.size.height, 0);
      transform = CGAffineTransformScale(transform, -1, 1);
      break;
  default:
      break;

}
// Now we draw the underlying CGImage into a new context, applying the transform
// calculated above.
CGContextRef ctx = CGBitmapContextCreate(NULL, self.size.width, self.size.height,
CGImageGetBitsPerComponent(self.CGImage), 0,
CGImageGetColorSpace(self.CGImage),
CGImageGetBitmapInfo(self.CGImage));
CGContextConcatCTM(ctx, transform);
switch (self.imageOrientation)
{
case UIImageOrientationLeft:
case UIImageOrientationLeftMirrored:
case UIImageOrientationRight:
case UIImageOrientationRightMirrored:
// Grr...
CGContextDrawImage(ctx, CGRectMake(0,0,self.size.height,self.size.width), self.CGImage);
break;
  default:
      CGContextDrawImage(ctx, CGRectMake(0,0,self.size.width,self.size.height), self.CGImage);
      break;

}
CGImageRef cgimg = CGBitmapContextCreateImage(ctx);
UIImage *newImage = [UIImage imageWithCGImage:cgimg];
return newImage;
}

I was able to solve my issue. I am using the cocoa pod and the pod was not
updated with the latest version of code. In particular this function in
TWPhotoPickerController was not reflected in the cocoa pod.

(void)collectionView:(UICollectionView *)collectionView
didSelectItemAtIndexPath:(NSIndexPath *)indexPath

{
ALAsset * asset = [self.assets objectAtIndex:indexPath.row];

UIImage *image = [UIImage imageWithCGImage:asset.defaultRepresentation.

fullResolutionImage scale:asset.defaultRepresentation.scale orientation:(
UIImageOrientation)asset.defaultRepresentation.orientation];
[self.imageScrollView displayImage:image];

if (self.topView.frame.origin.y != 0) {

    [self tapGestureAction:nil];

}

}
On Sat, May 16, 2015 at 9:06 AM, Krystel Chaccour<EMAIL_ADDRESS>wrote:

I am experiencing the same issue @codeSupreme
https://github.com/codeSupreme.
Were you able to fix it or have a workaround?
Thanks a lot
—
Reply to this email directly or view it on GitHub
https://github.com/wenzhaot/InstagramPhotoPicker/issues/11#issuecomment-102624859
.

Thanks @pushtheworldllc and @codeSupreme
It was actually the same problem @codeSupreme faced, I'm using pods as well.
And updating "didSelectItemAtIndexPath" method solved it for me :)
Thank you !

I also experienced this bug. @wenzhaot will you update your pods to fix this bug?

Thanks @codeSupreme!
Your solution fixed my problem!
@wenzhaot try solution above it works at least for me and @Krystel.
Best wishes!

Thanks @codeSupreme!


fixed bridge auth config accepts tenant ID as empty

This is helpful when you want to pass through requests that are not limited to a particular tenant.

Codecov Report

Merging #8 (c0bc25c) into main (99bf9c0) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main       #8   +/-   ##
=======================================
  Coverage   53.92%   53.92%
=======================================
  Files          14       14
  Lines         625      625
=======================================
  Hits          337      337
  Misses        268      268
  Partials       20       20

Impacted Files
Coverage Δ

internal/auth/auth.go
85.18% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 99bf9c0...c0bc25c. Read the comment docs.


Supportability on jaco.js

We are using jaco.js to implement Japanese transliteration solution. Hence, we would like to check with you on the following:-

If we have enquiries/concerns when using jaco.js, how can we raise our questions? How soon we can get respond from you?
If we encounter errors/issues when using jaco.js, how can we get support on this matter?
If there is a new release on this library, how can we get notified so that we can test on it?

Hello pckoay,
I'm Saphryn, and I've recently been put in charge to handle all English correspondence.
I've translated your post and will discuss your questions with the dev team today.
Thank you for your patience. You'll hear back from us very soon!


H2GIS driver

This PR implements H2GIS driver. It is outlined in my comment to  #6557
Regards

Dear @serge-rider ,
If this PR is okay for you, you can merge it.
For the other developments, I will exchange with you by email.
Best

I think this new extension should extend default H2 meta model. At least to avoid copy-paste in H2GISMetaModel.
Also there is still no functionality related to Gis data viewer.
I think that one of main points of this extension should be a possibility to view Gis data like in PG/MySQL:

Please write me to discuss details of this.


Is there recommend?

When I checked TF-recomm, I couldn't find out recommend api.
As you know, movie recommend engine can recommend movie like crab recommender.

#Recommend items for the user 5 (Toby)
recommender.recommend(5)
[(5, 3.3477895267131013), (1, 2.8572508984333034), (6, 2.4473604699719846)]

Is there any recommend api like above example?

The purpose of this repo is to inspire people that you can apply Tensorflow to non-deep learning problems and use a unified framework to solve various projects. Therefore, the repo only focuses on the core ML parts rather than serving parts. In my option, it wouldn't be hard to extend the core algorithm to better serving APIs. However, this is not in my top interests.


Feature - Skip words or Regex

Allow for certain calendar items to be skipped using stop words/sentences or regex

Can I know what the usage scenarios of this feature are?

@haydenull this would be for placeholder stuff people use in thier calendar or not real items they need to put notes/agenda against. Again a simple-ish thing to add to reduce noise in LogSeq for large calendars like mine.


ktformat not removing space around .

Before:
val a = "" .toString()

After:
val a = "" .toString()

Expected:
val a = "".toString() // no space between "" and .

@shyiko I believe this one could be closed by #293

Right :) Thanks for pointing it out!


Docs copy-paste mistake with _.unzip function

There is a misinformation about "unzip" function in the docs. Seems like a example was copied from "zip" description and someone forgot to change this.
Now it is:
_.unzip([['moe', 'larry', 'curly'], [30, 40, 50], [true, false, false]])
=> ["moe", 30, true], ["larry", 40, false], ["curly", 50, false]
Should be (The opposite of "zip" function):
_.unzip([["moe", 30, true], ["larry", 40, false], ["curly", 50, false]])
=> ['moe', 'larry', 'curly'], [30, 40, 50], [true, false, false]
Cheers!

Feel free to open a pull!

Done!

The Docs page is still showing the incorrect example for unzip
Should I pull and fix?

Sorry about that, it was pulled into master but not gh-pages. It should be live on underscorejs.org now.

checks out great, thanks.


Tabs: remove is prefix in props

Closes #3498
Description
Removing all is prefix from is*
Current behavior (updates)
Before: isFitted, isLazy, isManual
New behavior
After: fitted, lazy, manual
Is this a breaking change (Yes/No):
No
Additional Information

@hirotomoyamada
All suggested changes have been applied! Please review!


suppression rule for #1622

Fixes Issue #1622
Description of Change
Ignore false positive for org.openjdk.jmh:jmh-core:*.
Have test cases been added to cover the new functionality?
no

Thanks for the PR.  I did not see this and created the identical rule today.

No problem, it happens - I just shared what I done for myself :)


Enable rails 6.1

Resolves #1443

@ MaicolBen -
FWIW - I'm seeing this break bundle on the project in master, with "Resolving dependencies" looping indefinitely. If bundle is run with the DEBUG_RESOLVER=1 flag, it produces the following with the following trace (over and over):
Creating possibility state for rails (>= 4.2.0, < 6.2) universal-darwin-20 (66 remaining)
Attempting to activate [rails (<IP_ADDRESS>) (universal-darwin-20)]
Activated rails at [rails (<IP_ADDRESS>) (universal-darwin-20)]
Requiring nested dependencies (actioncable (= <IP_ADDRESS>) universal-darwin-20, actionmailer (= <IP_ADDRESS>) universal-darwin-20, actionpack (= <IP_ADDRESS>) universal-darwin-20, actionview (= <IP_ADDRESS>) universal-darwin-20, activejob (= <IP_ADDRESS>) universal-darwin-20, activemodel (= <IP_ADDRESS>) universal-darwin-20, activerecord (= <IP_ADDRESS>) universal-darwin-20, activesupport (= <IP_ADDRESS>) universal-darwin-20, bundler (>= 1.3.0) universal-darwin-20, railties (= <IP_ADDRESS>) universal-darwin-20, sprockets-rails (>= 2.0.0) universal-darwin-20, Ruby (>= 2.2.2) universal-darwin-20, RubyGems (>= 1.8.11) universal-darwin-20)
Creating possibility state for RubyGems (>= 1.8.11) universal-darwin-20 (1 remaining)
  Attempting to activate [RubyGems (3.0.3) (ruby), RubyGems (3.0.3) (universal-darwin-20)]
  Found existing spec ([RubyGems (3.0.3) (ruby), RubyGems (3.0.3) (universal-darwin-20)])
  Creating possibility state for bundler (>= 1.3.0) universal-darwin-20 (2 remaining)
    Attempting to activate [bundler (2.2.3) (universal-darwin-20)]
    Found existing spec ([bundler (2.2.3) (universal-darwin-20)])
    Creating possibility state for actionpack (= <IP_ADDRESS>) universal-darwin-20 (2 remaining)
      Attempting to activate [actionpack (<IP_ADDRESS>) (universal-darwin-20)]
      Found existing spec ([actionpack (<IP_ADDRESS>) (universal-darwin-20)])
      Unsatisfied by existing spec ([actionpack (<IP_ADDRESS>) (universal-darwin-20)])

If I revert the Gemspec changes, Bundle resolves correctly ...

^ FYI @MaicolBen (sorry if this is pinging you twice, I noticed the mention in the previous comment wasn't formatted correctly)
For anyone that runs into this issue when setting up the project locally, you can temporarily change the devise_token_auth.gemspec file's s.add_dependency 'rails' line to be s.add_dependency 'rails', '>= 4.2.0', '< 6.1'. Note the 6.2 to 6.1 tweak.

^ FYI @MaicolBen (sorry if this is pinging you twice, I noticed the mention in the previous comment wasn't formatted correctly)
For anyone that runs into this issue when setting up the project locally, you can temporarily change the devise_token_auth.gemspec file's s.add_dependency 'rails' line to be s.add_dependency 'rails', '>= 4.2.0', '< 6.1'. Note the 6.2 to 6.1 tweak.


sys-kernel/coreos-modules: Enable CONFIG_GTP

sys-kernel/coreos-modules: Enable CONFIG_GTP
This is the GPRS Tunneling Protocol datapath for usage in telecoms scenarios. It has been requested by a user.
How to use
We probably can't use this kernel module ourselves, but we should at least check that it loads.
Testing done
I'll run the GHA pipeline and check the image.

[X] Changelog entries added in the respective changelog/ directory (user-facing change, bug fix, security fix, update)
[ ] Inspected CI output for image differences: /boot and /usr size, packages, list files for any missing binaries, kernel modules, config files, kernel modules, etc.

Hello @tormath1
I'm the initiator of this feature request, after checking on my side, the module seems to weigh something like 16K


Add unit test to OfferCardSquare and fix err / bug in component

Add unit test to OfferCardSquare and fix err / bug in component

as I see, the test coverage for OfferCardSquare is 100%
Is this task still relevant?

@ArturBekhDEV could you clarify the bug which should be fixed? and if it is still relevant as well.


Cityscapes experiment

Hi,
Thanks for open sourcing the code, this is great!
Could you share your json parameter file for cityscapes?
Also, I think it is is missing the file depth_mean.npy to be able to run it.
Thanks.

We will update the code in a few days with depth_mean.npy and config files. But, in the mean time here are the config files if you do not want to wait:

Parameter Set w/ Approximation:
optimizer=Adam|batch_size=8|lr=0.0005|dataset=cityscapes|normalization_type=none|algorithm=mgda|use_approximation=True

Parameter Set w/o Approximation:
optimizer=Adam|batch_size=8|lr=0.0001|dataset=cityscapes|normalization_type=none|algorithm=mgda|use_approximation=False

depth_mean.npy is the average depth map of training set. We use it for making the input zero mean.

Thanks. Will the update code be running with pytorch 1.0? I am getting into a few problems to run it since some features are deprecated (e.g. volatile variables, .data[0], etc.)

I am also having an error with the FW step:
sol, min_norm = MinNormSolver.find_min_norm_element([grads[t] for t in tasks])
----> 1 sol, min_norm = MinNormSolver.find_min_norm_element([grads[t] for t in tasks])

~/MultiObjectiveOptimization/min_norm_solvers.py in find_min_norm_element(vecs)
     99         # Solution lying at the combination of two points
    100         dps = {}
--> 101         init_sol, dps = MinNormSolver._min_norm_2d(vecs, dps)
    102
    103         n=len(vecs)

~/MultiObjectiveOptimization/min_norm_solvers.py in _min_norm_2d(vecs, dps)
     42                     dps[(i, j)] = 0.0
     43                     for k in range(len(vecs[i])):
---> 44                         dps[(i,j)] += torch.dot(vecs[i][k], vecs[j][k]).data[0]
     45                     dps[(j, i)] = dps[(i, j)]
     46                 if (i,i) not in dps:

RuntimeError: dot: Expected 1-D argument self, but got 4-D

What exactly should containgrads and [grads[t] for t in tasks]?

I can confirm that those changes worked for me to get the code running with pytorch 1.0.
I was able to reproduce the results for the single task models, but so far no luck with the mgda method.
Did you have to include any other changes @maria8899

@r0456230 Can you tell me what exactly you are trying to reproduce? The config files I put as a comment should give exact results of mgda w/ and w/o approximation.
Please note that; we report disparity metric in paper and compute depth metric in the code. Depth map is later separately converted into disparity as post-processing. If the issue is depth, this should explain it.
mIoU should be exactly same with what reported in the code and the paper. We used the parameters I posted as a comment.

@maria8899 Although we are planning to support pytorch 1.0, I am not sure when will it be. I will also update the ReadMe with the exact versions of each Python module we used. Pytorch was 0.3.1

@ozansener I was able to reproduce the results from the paper for the single task models using your code (depth, instance segmentation and semantic segmentation on cityscapes).
However, when I run the code with the parameters posted above, after 50 epochs the models seems to be far removed from the results obtained in the paper.

I think I have manged to make it work with pytorch 1.0, but I still need to check the results and train it fully.
@r0456230 I haven't done much other changes, the main problem was in this FW step.

@maria8899 @r0456230 could you please tell me how to solve depth_mean.npy missing problem? I try to reproduce the cityscapes results too.

@YoungGer you need to compute a mean image (per pixel) using all training images (or just a few to get an approximation) of the cittyscapes's disparity dataset.

@YoungGer you need to compute a mean image (per pixel) using all training images (or just a few to get an approximation) of the Cityscapes's disparity dataset.

I know, thank you for your help.

@YoungGer Have you noticed, that the find_min_norm_element method actually uses the projected gradient descent method? Only find_min_norm_element_FW is the Frank-Wolfe algorithm as discussed in the paper. They are only guaranteed to be equivalent for a number of tasks equal to 2.

Hi @ozansener ,
thanks for publishing your code!
I have two questions after reading this answer by @maria8899

Edit: the solution is to replace with
torch.dot(vecs[i][k].view(-1), vecs[j][k].view(-1)).item()

In your code the z variable returned by the 'backbone' of the network is passed to each task.
Its gradient is then used in the find_min_norm algorithm.
First of all as maria noted, the gradient is a 4D variable that needs to be reshaped first to 1D in torch 1.0.1.
I compared the behavior to torch 0.3.1 and it does lead to the same result, but it raised some questions, which very well might come from my missing understanding of your paper.

The gradient still has the batch dimension, why do you calculate the the min_norm_point between all samples as one big vector instead of for example averaging or summing over the batch dimension? Isn't this what is effectively happen after the reshaping? This is just intuitively speaking comparing it with stochastic gradient descent.
Why is there a batch dimension anyway? From the paper it is not quite clear to me what should be fed into the FrankWolfeSolver, but shouldn't it be the gradient of some parameters instead of an output variable? Or does that not matter and lead to the same result?

Thanks a lot!

@kilsenp First let me answer the 2. You are right if you apply MGDA directly, it should be gradients with respect to parameters. However, one of the main contributions of the paper is showing that instead you can actually feed gradients with respect to the representations. This is basically the Section 3.3 of the paper and what we are computing in the code is $\nabla_Z$. 1. No, you need batch dimension since forward pass of the network is different for each image. You can read Section 3.3 of the paper in detail to understand whats going on.

Hi, @maria8899, @kilsenp,
Have you reproduced the results on MultiMNIST or CityScapes? Thanks.

Hi,@liyangliu ,
Have you reproduced the results on MultiMNIST? I have tried but only got the result like grid search.Would you like to tell me the params you chosen?Thanks.

@youandeme, I haven't reproduced the results on MultiMNIST. I used the same hyper parameters mentioned by the author in https://github.com/intel-isl/MultiObjectiveOptimization/issues/9, but can not surpass the uniform scaling baseline. Also, I noticed in the "Gradient Surgery" paper (supplementary materials), other researchers report different results on MultiMNIST from this MOO paper. So I doubt that others also have difficulty in reproducing the results on MultiMNIST following this MOO paper.

@liyangliu  @youandeme  How are you evaluating the MultiMNIST? We did not release any test set, actually there is no test set. The code generates random test set every time you run. For all modules, you simply use the hyper-params I put. Then, you save every epoch result and choose the best epoch with the best val accuracy. Then, you call MultiMNIST test which will generate random test set and evaluate it. If you call the MultiMNIST loader with test param, it should do the trick. If you evaluate this way, the result are not exactly matching since test set is randomly generated, but the order of methods is preserved.

Hi, @ozansener, as you mentioned, the order of different methods (MGDA-UB vs. uniform scaling) will keep the same whatever test set I use. But on the validation set, I cannot find the superiority of MGDA-UB upon uniform scaling. Also, on CityScapes I cannot reproduce the results reported in the paper. Actually I find that single-task baseline is better than the reported ones (10.28 vs. 11.34, 64.04 vs. 60.68 on the semantic and instance segmentation task respectively). I obtain these numbers with your provided code, so maybe I made some mistakes?

@liyangliu For MultiMNIST, I think there are issues since we did not release a test. Everyone reports slightly different numbers. In hindsight, we should have released the test set but did not even save it. So, I would say please report whatever number you obtained for MultiMNIST. For Cityscapes though, it is strange as many people re-produced the numbers. Please send me an e-mail about the CityScapes so we can discuss.

Thanks. @ozansener. On CityScapes I re-run your code with instance & semantic segmentation and get the following results for MGDA-UB and SINGLE task, respectively:

method
instance
semantic

MGDA-UB
15.88
64.53

SINGLE
10.28
64.04

MGDA-UB (paper)
10.25
66.63

SINGLE (paper)
11.34
60.08

@liyangliu Instance segmentation one looks strange. Are you using the hyper-params I posted for both single task and multi-task. Also, are the tasks uniformly scaled or are you doing any search. Let me know the setup.

@ozansener, I use exactly the hyper-params you posted for single & multi-task training. I use 0.5 & 0.5 for instance and semantic segmentation and do not do any grid search.

Sorry for an off-topic question, but I have trouble even running the training on CityScapes: for 256x512 input I get 32x64 output, while the target is 256x512. And smaller output makes sense to me because of the not dilated earlier layers & maxpooling.
So could someone please clear up for me, whether target indeed should have the same dimensions as input, and if so, where the spatial upsampling is supposed to happen?

@AwesomeLemon @ozansener if you know the answer, can you please comment on it?


Onboarding flow for fiscal hosts to customize

Who is your user?
Fiscal Host Admins
What are they trying to achieve?
Enable fiscal hosts to customize their own onboarding flow on platform. Most fiscal hosts have a due diligence process they need to follow, certain questions or processes that need to be checked off like an onboarding call or 2 admins.
When onboarding a new collective, I want to be able to customize the onboarding flow, so I can keep track of where new groups are in the onboarding process
How are they currently doing this?
Using a range of third party tools
How well understood is this problem?

[ ] Surfacing: if you feel this is a seed of an idea that needs to be researched.
[ ] Understanding: if you've done some research and feel it is already well understood and ready for shaping.
[ ] Shaping:  if you feel a solution is clearly defined and can be made into a pitch (read more about shaping)
[ ] Pitching: if you feel that it is ready to be presented for prioritisation (read more about pitching)

Due diligence - Vibes - formalize the gut feeling about groups - use machine learning to help with this process


Extend Snapshot.page from 1:1 voting to Quadratic Voting

Actual behavior
1:1 voting
Expected behavior
Implement quadratic voting on snapshot.page
WHY: Quadratic Voting ( https://en.wikipedia.org/wiki/Quadratic_voting ) is a mathematically elegant way to make sure that decisions are made in a more democratic way.  It mutes the influence of whales. It optimizes for the voting power of the "poor and many" over the "rich and many".
In order to implement QV on Snapshot.page, i think all we need to do to do an entirely trivial QV implementation is:

For each new proposal, the user can select '1:1 voting' or "QV voting".
If the proposal is in QV mode, votes on that proposal should count quadratically instead of 1to1.  (the math is easy; just take the sqrt of the vote instead of the vote, and add that to the total - you can see an example here https://wtfisqf.com/ - esp in the linked javascript library )

A less naive implementation (lets call it Milestone 2) would be to introduce some sort of sybil resistence to the system.  We’d have to integrate with something that can provide Sybil resistance as an minimal identity layer...  Let's cross that bridge when we get to it, but some early promising routes

BrightID integration (make each voter be a verified BrightID user), and use that as sybil resistence.
Integrate with Gitcoin (make each voter authenticate with their Gitcoin account), and use that as sybil resistence.

Awesome!
For the milestone 1 i recommend to use a "strategy". Strategies are functions that calculate result of a proposal on the UI. The default strategy is to take the balance of an ERC20 for each voters. We can duplicate the erc20-balance-of strategy maybe call it erc20-balance-of-qv and apply a sqrt() of the results at the end of the function. Learn more here about strategies here: https://docs.snapshot.page/strategies

I added radio buttons for view type options to the interface, would need to coordinate more to make sure end point support this feature.

Quadratic voting in non Sybil resistant environments actually amplifies the voting power of bad actors tenfold. Be careful.

Hence why BrightID my friend. Gitcoin has done an insane amount of work to improve the sybil resistance of their protocol since round 1.

@bonustrack would you be ok if i bountied this out to get it built? i think we cna use brightid for the anti sybil layer

Would also be interested in seeing this built.

Would this strategy https://github.com/snapshot-labs/snapshot.js/blob/master/src/strategies/erc20-balance-of-cv/index.ts be Milestone 1?


installing eosio on mojave 10.14.6 not working from prebuilt binaries

Hi, I followed instructions to install eosio with prebuilt binaries and it gives me this message. So I tried to build the binaries as an experienced developer and thats not working perhaps because I am not experienced enough in mac platform, homebrew and terminal. The message I get when trying the install with pre-built binaries is this. Can somebody help with this issue? Any suggestions? My system is a 2017 imac mojave 10.14.6 using updated of homebrew

Thank you

My hunch is that this will be resolved in 2.1.1 since in 2.1.1 there is no longer a dependency on libpqxx. Not sure when 2.1.1 will be released though.


Add useful links

Adding useful links for the things mentioned in this issue: https://github.com/CodingTrain/Logo/issues/27

Awesome, thank you for this!


Rendered version of DCO at https://probot.github.io/apps/dco/ contains unmatched references to (a), (b) and (c)

https://probot.github.io/apps/dco/ says:
By making a contribution to this project, I certify that:

1. The contribution was created in whole or in part by me and I have the right to submit it under the open source license indicated in the file; or

2. The contribution is based upon previous work that, to the best of my knowledge, is covered under an appropriate open source license and I have the right under that license to submit that work with modifications, whether created in whole or in part by me, under the same open source license (unless I am permitted to submit under a different license), as indicated in the file; or

3. The contribution was provided directly to me by some other person who certified (a), (b) or (c) and I have not modified it.

4. I understand and agree that this project and the contribution are public and that a record of the contribution (including all personal information I submit with it, including my sign-off) is maintained indefinitely and may be redistributed consistent with this project or the open source license(s) involved.

The paragraphs are numbered with 1., 2., 3, and 4. but the refernce in 3. is (a), (b) or (c).

It is still wrong at https://probot.github.io/apps/dco/ and it is blocked by me not having write access to the system to change the layout or someone else caring to change it.

PRs welcome: https://github.com/probot/probot.github.io/blob/master/_apps/dco.md

PRs welcome: https://github.com/probot/probot.github.io/blob/master/_apps/dco.md

@hiimbex what did you mean with "PRs welcome"? I filed the trivial PR 23 days ago and got no feedback so far... IMHO this is not how to welcome PRs.

I'm no longer at GitHub and their team supporting this probot has shrunken/shifted priorities. I do try to merge DCO things when I can. Sorry for the inconvenience


Documentation

Create documentation regarding the usage, installation and also improve doc strings.

Documentation may be improved in future. For now it is ok and anyone can use.


[Need Info] Need information about styles

Hi,
Where do i find different types of styles that can be passed to "style" attribute in JSON?
"style":"xt256", //set highlight
Thanks,
Santosh

hi Santosh，
you can find different type in  https://highlightjs.org/


Remove and Archive OSM (Open Service Mesh) from Meshery

Currently, osm has been archived and is no longer in development.
Please see opened issue at meshery-osm: https://github.com/meshery/meshery-osm/issues/246
We will need to remove to every instance of meshery-osm and osm related.

[ ] Archive https://github.com/meshery/meshery-osm
[ ] Remove OSM from SMP proto
[ ] Remove OSM from golden template in mesheryctl
[ ] Remove OSM from default meshconfig
[ ] Remove OSM from docker extension

Contributor Guides and Handbook

🛠 Meshery Build & Release Strategy
📚 Instructions for contributing to documentation

Meshery documentation site and source

🎨 Wireframes and designs for Meshery UI in Figma
🙋🏾🙋🏼 Questions: Discussion Forum and Community Slack

@nebula-aac can you assign this issue to me ?

Please assign this to me

Can you assign me this issue ?

Folks, this isn't a good first issue. @nebula-aac or @Revolyssup will you be carrying this forward?


doc/compression.md: Make it clear compression is per-RPC

"initial call" should not be understood as "initial call on the connection" but
"initial call of the RPC". All the data necessary for understanding client's
compression abilities is provide per-RPC, so no connection-level state should
be used. Using such state would break the use of proxies.

@dfawley, friendly ping. Pretty small change.


Cloudfront hosting: invalid default cache behaviour for deployed web sites

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
14
Amplify CLI Version
7.6.5
What operating system are you using?
Mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No change
Amplify Categories
hosting
Amplify Commands
add, configure
Describe the bug
When using Amazon CloudFront and S3 as a hosting, a lot of js and css files are fingerprinted (they have a unique names). Some other files like index.html are not. For these files, as their name will not change from one deploy to another but their content will, we need to consider the browser's cache behaviour in this case.
Currently there is no way to set a cache header for specific files when using Amazon CloudFront and S3. There is one way when we host via the Amplify Web console here: https://docs.aws.amazon.com/amplify/latest/userguide/custom-headers.html
What happens around browser cache behaviour when cache headers are not provided? Browsers will then use a heursitic based on the file last modified time. Some more information is available here: https://docs.aws.amazon.com/amplify/latest/userguide/custom-headers.html
It is safe to say that without the cache control header, the behaviour is not deterministic. Suppose a user came to your website a day ago with index.html having no cache-control headers set. When he came, the last modified date for this file was 1 month ago. Today you are publishing an update and the user comes back to the website. The browser will keep caching the old index.html file for the next 3 days, at least. The old index.html file has references to css and js that are outdated. That means your whole website will be outdated to this user for the next three days.
Expected behavior
We would expect that when using Amazon CloudFront and S3 as a hosting, the created architecture would have sensible defaults (or bug free behaviour) when it comes to caching.
Two things would enable this:

Having the AWS Amplify CLI considering the custom headers as well for Amazon CloudFront and S3 hosting. We imagine this would mean that during amplify publish the CLI would look through the amplify.yml CustomHeaders section and push the file to S3 with the correct meta-data. Someone took a stab at it already by the look of it: https://github.com/aws-amplify/amplify-cli/issues/6393 , PR: https://github.com/aws-amplify/amplify-cli/pull/6879 while other users also ask for this behaviour here: https://github.com/aws-amplify/amplify-cli/issues/167
Having sensible customHeaders patterns provided by default in amplify.yml or in documentation.

Together, these two things would allow amplify users to add a Amazon CloudFront and S3 hosting without having to deal with weird cache behaviours. Currently the only way avaiable we found to deal with these kind of issues is to build some custom scripts that hook into the deployment process, customizing the files' headers after deploy is done. Clearly, the framework could take ownership of this (like it does when deploying to Amplify Console).
Reproduction steps

Add a Amazon CloudFront and S3 hosting solution.
See that adding CustomHeaders in amplify.yml is not used to set the metadata of the files on S3.

GraphQL schema(s)
No response
Log output
No response
Additional information
No response

The inability to configure caching behavior and the presence of default settings that could lead to users receiving stale data for some time are deal-breakers for me. These kinds of issues make me think twice about considering AWS Amplify for a project.
When deploying a website with CDK, the index.html caching behavior is one of the things I have to set up.


[Telemetry] Standardize logging framework across all modules

Objective
Unify and standardize the logging approach across the entire codebase.
Origin Document
See the github discussion for full details: https://github.com/pokt-network/pocket/discussions/214
The implementation of this was started by @derrandz here: https://github.com/pokt-network/pocket/pull/162.
This ticket can either build on top of the existing WIP or use those learnings and apply them to a different approach.
Goals / Deliverables

[x] Research and identify common logging frameworks available in Go and use in major codebases (e.g. Uber)
[ ] Design a new shared logging module based on the results from the research above. This means it could be:

Created from scratch
Imports an existing framework
Extends an existing one with a lightweight wrapper

[ ] Refactor the entire codebase to use the same logging framework
[ ] Add support for multiple log levels
[ ] Add support for attaching labels to logs
[ ] Build a log panel in Grafana where logs can be toggled based on level or label

General issue checklist

[ ] Update the appropriate CHANGELOG
[ ] Update the appropriate README on how to use and test this feature
[ ] Update any relevant global documentation & references
[ ] Add a video and/or gif on how to use feature
[ ] Document small issues / TODOs along the way

Non-goals

Add new logs throughout the codebase
Improve the explicitness or correctness of existing logs
Add new business logic functionality

Testing Methodology

All tests: make test_all
LocalNet: verify a LocalNet is still functioning correctly by following the instructions at docs/development/README.md
Remove

Creator: @Olshansk
Co-Owners: @vlad-she @derrandz

Note that there will be cross-dependencies between this and https://github.com/pokt-network/pocket/issues/118

A WIP implementation of this is available here: https://github.com/pokt-network/pocket/pull/162

THis is blocked by https://github.com/pokt-network/pocket/issues/181

Split by

Design
Implementation/support

Remove

Grafana (cover in #189)

I started working on this, will post a PR soon.


Improper radio behavior on Publish

The Issue
When publishing content, the "Price" and "Identity" sections shows undesired behaviour. If the "Choose Price" radio is clicked once, and then "Free" is clicked, the "Price" text box is not hid. Same occurs when selecting "New Identity".
Steps to reproduce
See video.
Expected behaviour
Appropriate text fields should be hidden.
Actual behaviour
The text fields are not hidden.
System Configuration

LBRY Daemon version: 0.14.1
LBRY App version: almost latest commit from git, 902b7f2
Operating system: Linux

Video

#358 fixes this issue, IMO. This can be closed.


Microsoft.Web/certificates - Certificates - List: doesn't always return the correct resourcegroup matching the resource group in the resourceid

API Versions: 2019-08-01, 2020-12-01, 2021-01-01
Document Links:
https://docs.microsoft.com/en-us/rest/api/appservice/certificates/list
https://docs.microsoft.com/en-us/rest/api/appservice/certificates/list-by-resource-group
I have 9 resources of type "Microsoft.Web/certificates" each in a different resource group but a single subscription. When I make the API call to
GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Web/certificates?api-version=2019-08-01
or to
GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/certificates?api-version=2019-08-01
The JSON data describing a certificate object or a collection of certificate objects should have the same resource group name in the properties "id" and "properties.resourceGroup", but out of 9 resources, 7 have a resource group name in "properties.resourceGroup" which matches the resource group name from "id" and 2 have a resource group name for "properties.resourceGroup" which doesn't match the resource group name from "id".
I've tested the API calls Postman using each of the API versions listed at the top, using the call to "Get all certificates for a subscription".
I've tested both the API calls for "Get all certificates for a subscription" and "Get all certificates in a resource group" using the Try it feature from the documentation links above.
Originally I was using API Version 2019-08-01 from a logic app.
Each of these returns a "properties.resourceGroup" value for some resources which are incorrect when compared with the value for "id" (resourceId). The only way I've found I can get the correct value in JSON format, is when using resourceExplorer from the Azure Portal. This seemed to me to be making the same API call with the same version, though I can only see as far as the URL e.g.
url: "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Web/certificates/{name}?api-version=2020-12-01"
Except now I can see that it's making the call for "Get a certificate". So the issue appears to be present in Get all certificates for a subscription, Get all certificates in a resource group.
The call for "Get a certificate" seems to be the only call which returns the correct data.

@leni-msft might this be related to #1463 as the two affected API calls are calls which return certificate collections? 1463 is an old issue, but yet still open. I have no idea how long my issue has been present, but it could easily have been an issue back in 2017/2018. cc @azureappservicecli

After looking at #1463 I've looked further at the following two links to check the JSON swagger matches with the output of the API calls I make on API Version 2021-01-01.
https://github.com/Azure/azure-rest-api-specs/blob/master/specification/web/resource-manager/Microsoft.Web/stable/2021-01-01/Certificates.json
https://github.com/Azure/azure-rest-api-specs/blob/master/specification/web/resource-manager/Microsoft.Web/stable/2021-01-01/CommonDefinitions.json
An example Certificate definition taken from an array of Certificates JSON returned by the API Call I made to list all Certificates in a Subscription looks like:
{
    "id": "/subscriptions/00a00000-000a0-0a00-0000-a00000a0aa00/resourceGroups/AZE-RG-APP-ENV-API-FUNCTION-01/providers/Microsoft.Web/certificates/WildcardExampleCom",
    "name": "WildcardExampleCom",
    "type": "Microsoft.Web/certificates",
    "location": "West Europe",
    "tags": {
        "CONTACT"<EMAIL_ADDRESS>        "ENVIRONMENT": "ENV",
        "PRODUCT": "APP",
        "CRITICALITY": "TIER3"
    },
    "properties": {
        "friendlyName": "*.example.com",
        "subjectName": "*.example.com,www.example.com,www01.example.com",
        "hostNames": [
            "*.example.com",
            "www.example.com",
            "www01.example.com"
        ],
        "pfxBlob": null,
        "siteName": null,
        "selfLink": null,
        "issuer": "GlobalSign RSA OV SSL CA 2018",
        "issueDate": "2021-05-01T00:01:02+03:00",
        "expirationDate": "2022-05-30T02:03:04+03:00",
        "password": null,
        "thumbprint": "AB1234CDEF1AB1C123456A12BAC123A1A2C2A1B2",
        "valid": null,
        "toDelete": null,
        "cerBlob": null,
        "publicKeyHash": null,
        "hostingEnvironment": null,
        "hostingEnvironmentProfile": null,
        "keyVaultId": "/subscriptions/00a00000-000a0-0a00-0000-a00000a0aa00/resourcegroups/aze-rg-shared-prd-certificates-01/providers/microsoft.keyvault/vaults/kvtprdcertificates",
        "keyVaultSecretName": "WildcardExampleCom",
        "keyVaultSecretStatus": "Succeeded",
        "webSpace": "AZE-RG-APP-ENV-ASP-01-WestEuropewebspace",
        "serverFarmId": null,
        "tags": {
            "CONTACT"<EMAIL_ADDRESS>            "ENVIRONMENT": "ENV",
            "PRODUCT": "APP",
            "CRITICALITY": "TIER3"
        },
        "resourceGroup": "aze-rg-app-env-api-function-01"
    }
},

When I review this against the swagger, I find the swagger for a combination of Certificates and CommonDefinitions looks like the following list and then I've indicated which are defined or not and which are returned. That list looks like:
"$ref": "./CommonDefinitions.json#/definitions/Resource"
https://github.com/Azure/azure-rest-api-specs/blob/18aa5fc0a4aeb8d98b47d06c45ef5f551f9b7a81/specification/web/resource-manager/Microsoft.Web/stable/2021-01-01/CommonDefinitions.json#L2037
id (Optional) - Yes, value returned
name (Optional) - Yes, value returned
kind (Optional) - No
type (Optional) - Yes, value returned
location (Required) - Yes, value returned
tags (Optional) - Yes, value returned
"$ref": "#/definitions/Certificate"
https://github.com/Azure/azure-rest-api-specs/blob/0f72d7e8267d012ed7f51107f066217c8bfda9a7/specification/web/resource-manager/Microsoft.Web/stable/2021-01-01/Certificates.json#L310
"friendlyName" - Yes, value returned
"subjectName" - Yes, value returned
"hostNames" - Yes, value returned
"pfxBlob" - Yes, null
"siteName" - Yes, null
"selfLink" - Yes, null
"issuer" - Yes, value returned
"issueDate" - Yes, value returned
"expirationDate" - Yes, value returned
"password" - Yes, null
"thumbprint" - Yes, value returned
"valid" - Yes, null
"toDelete" - Yes, null, but not in the Certificates.json swagger file
"cerBlob" - Yes, null
"publicKeyHash" - Yes, null
"hostingEnvironment" - Yes, null, but not in the Certificates.json swagger file
"hostingEnvironmentProfile" - Yes, null
"$ref": "./CommonDefinitions.json#/definitions/HostingEnvironmentProfile"
https://github.com/Azure/azure-rest-api-specs/blob/18aa5fc0a4aeb8d98b47d06c45ef5f551f9b7a81/specification/web/resource-manager/Microsoft.Web/stable/2021-01-01/CommonDefinitions.json#L1206
"id" - No hostingEnvironmentProfile null
"name" - No hostingEnvironmentProfile null
"type" - No hostingEnvironmentProfile null
"keyVaultId" - Yes, value returned
"keyVaultSecretName" - Yes, value returned
"keyVaultSecretStatus" - Yes, value returned
"webSpace" - Yes but not in the Certificates.json swagger file
"serverFarmId" - Yes, null
"tags" - Yes but not in the Certificates.json swagger file
"resourceGroup" - Yes but not in the Certificates.json swagger file
"canonicalName" - No
"domainValidationMethod" - No
So it seems like some of my invalid data, the incorrect resourceGroup and also the tags as part of the Certificate object properties, aren't in the Certificates.json swagger file. Does this mean I shouldn't rely on these properties? Why wouldn't I rely on the data returned when I call a Microsoft Azure API? Is the API implementing something it shouldn't? Or is the swagger missing something which has been implemented in the API?

@petrpod would you be able to take a look at this issue or route to correct person? thanks!

@leni-msft will take a look at this & respond back. Assigning to myself for now.

@WillParry could you show us some responses from ListCertificates and ListCeritficatesByResourceGroup API with mismatching resource group please? Specifically what is the certificate thumbprint and what is the incorrect resource group showing in properties.resourceGroup?

@ElleTojaroon is it possible to share that internally? There is a new MS ticket raised for this issue now logged via our MSP / CSP. Reference is<PHONE_NUMBER>001310. I'm then more than happy to share it, otherwise to post publicly, I'd need to obfuscate the resource details etc. Thanks for responding / reaching out, apologies I didn't spot your response earlier.

@ElleTojaroon just updating that I believe the data was shared with you internally. If you haven't received it yet, please let me know. Thanks.


Always author year citation

The short citation in footnotes should always follow the author year style, not only when there are multiple citations of the same author.

If 85fa0e70574d618214a97cec339da7c78a969547 will be merged, this will be solved.


v2.1.0 updates

I looked through all the changes between v2.1.0 of luxai_s2 and v2.0.0 and these were the only ones that needed changing. You also actually fixed a bug that I only just fixed (collision handling when multiple heavies enter but all have lower power than a light unit there).

This change also means the old unit tests will probably fail. The v2.1.0 episodes have not begun running just yet but will soon


feat: discussion feature

Is your feature request related to a problem? Please describe.
I want to create issues on a resource so that other users can see it.
Describe the solution you'd like

CRUD Issue apis for a resource

Use cases
Discussion

Create a discussion
Get a discussion (filter by asset)
List discussion (filter by asset)
Update a discussion
Upvote discussion (out of scope)
Pinning/Unpinning discussions (out of scope)

Discussion Comments

List discussion comments
Get a discussion comment
Create discussion comment
Update discussion comment
Delete discussion comment
Upvote discussion comment (out of scope)

Discussion Labels

List labels of a discussion

Users' Discussions

List assigned discussion of a user

With use cases mentioned above, we could generalize it into discussions by defining it into multiple types.
Discussion Property

Title
Body
State
Assignees (nullable, array of string)
Assets (nullable, array of string)
Owner (not null)
Labels (nullable, array of string)
Type (int)

Discussion Types

Issues
states
-   Unknown: enum 0
-   Open: enum 1
-   Closed: enum 2
Q&A
states
-   Unknown: enum 0
-   Unanswered: enum 1
-   Answered: enum 2
Comments can be marked as an answer  (out of scope)
Suggest answer (out of scope)
Vote best answer (out of scope)
Open-ended questions
No states

Users discussion

Return discussion in user context
Owned, assigned

Comments

Comment belongs to the discussion
Discussion could have multiple comments
Comments could have replies which is another comments (out of scope)
Reply comment could not be upvoted (out of scope)


Updates to manifests descriptions

Couple of minor changes here for polish - let me know your thoughts!

Thanks @nathan-weinberg !
Could you run make bundle and add the generated bundle files to the commit?

/cc @jotak @jpinsonneau @mariomac

/lgtm
/approve


Create Rakefile to fix the build

Thank you for your contribution; we've actually furthered development on this project and have been able to add some passing tests.


Not working android build because of .fscache bug related to game assets update

Hey there..
If I add new resources to my project and then try to export the project to my android device the game does not start. Logcat shows that the newely added resources are not found.
Checking the .fscache file showed me, that the new resources were not added. Fix for that is deleting the .fscache and then restarting the project which creates a new .fscache.
This seems to be a write permission issue?

I believe it's fixed by now, if not please let me know I will reopen.


Minimal viable product | Iteration 1

Key specs for the MVP1

[x] Have as many species as possible listed in https://brc-analytics.org/data/organisms
[ ] Have all species from above registered as Galaxy Reference data with corresponding indices for bowtie (1 & 2), bwa-mem (1 & 2), STAR, hisat2, SNPeff
[ ] Have accounts implemented linking brc-analytics with Galaxy (https://github.com/galaxyproject/brc-analytics/issues/77): change add Google authentication to .org only for MVP1
[ ] Cross-linked for gene pages between NCBI and UCSC:

This UCSC page:

should link to a corresponding gene page at NCBI Datasets via Gene ID:

and vice versa

[ ] Make sure datasets can be displayed from Galaxy in a corresponding UCSC Genome Browser (https://github.com/galaxyproject/galaxy/issues/18901)
[ ] "Analyze" page for an organism (e.g., https://brc-analytics.org/data/organisms/GCA_000150785_1) should contain "Data portals" and "Analysis portals" sections as described here https://github.com/galaxyproject/brc-analytics/issues/106
[ ] When one goes to Galaxy from "Data portals" section of the analysis page for a given species, the Galaxy should be focused on that species (https://github.com/galaxyproject/galaxy/issues/18882)

A more detailed description of BRC Analytics / Galaxy connection for MVP1

Start on Organism page
Click "Analyze" button
This takes you to analysis page (e.g., https://brc-analytics.org/data/organisms/GCA_000826245_1)
On analysis page you:

choose which instanced (test or main at this time)
choose a workflow (workflow needs to be from IWC to have a TRS ID)
click "Analyze"

If you selected "test" -> you get workflows start interface pre-set to GCA_000826245_1
If you selected "main" -> you just get  workflows start interface and set dbKey manually (https://github.com/galaxyproject/brc-analytics/issues/121)


WpfPlot (control) does not appear in toolbox

Issue: After installing ScottPlot via PackageManager the UserControl "WpfPlot" does not appear in the Toolbox (VS 2019).
I found two solutions that might be mentioned in the Quickstart document to make life easier for a beginner:
Solution 1: Add Control manually in the XAML editor without adding the UserControl to the Toolbox:
 <Grid>         <scottplot:WpfPlot Name="wpfPlot1" Margin="10"/>         ...   </Grid>
Solution 2: Right-click the Toolbox, pick "Choose Items...", click "Browse" in any tab, find the ScottPlot.dll, select it and click "Open".
In both cases you have to add in the window tag:
xmlns:scottplot="clr-namespace:ScottPlot;assembly=ScottPlot"

Hi @RalphLAtGitHub, thanks for mentioning this! Although someone can use the WpfPlot control by manually creating it in text, having it appear in the toolbox would be much easier. I hadn't noticed it doesn't appear by default.
After some googling I think this is what I need to do to fix this issue: https://docs.microsoft.com/en-us/nuget/guides/create-ui-controls#add-toolboxassets-pane-support-for-xaml-controls
I've give it a shot and update this issue when I learn more

@RalphLAtGitHub thanks again for bringing this to my attention!
I couldn't easily figure out how to do this with the current packaging system. I'm about to use a new packaging system when we migrate to .NET Core 3.0 (#86) so I won't prioritize this for now.
Based on your tips I updated the WPF quickstart to clarify the toolbox issue.
https://github.com/swharden/ScottPlot/blob/master/doc/quickstart/README.md#wpf-application
Fingers-crossed this will fix itself after the migration to .NET Core 🤞

I found an open-source NuGet package that adds a custom control to the toolbox. No xmlns registration needed in your xaml files either. I'll see if I can mimic what they do...
https://github.com/dotMorten/UniversalWPF

I can't believe I spent 5 hours on this... and it's still not working 💀 So far I:

made a VisualStudioToolsManifest.xml and added a ToolboxItems section
edited the csproj file to:

include the xml manifest
set OutputType to library
set RootNamespace to ScottPlot

edited the assemblyinfo.cs to include XmlnsDefinition for ScottPlot

... but it's still not working. I have to put this down or none of these other open issues are going to get attention 🐌

Right after posing this I tired one more thing... and it works!
This whole time I was testing nuget packages locally. It seems visual studio gets confused if you don't increment the version number every time 🙈
Either way, I finally figured this one out. I'll release it on NuGet in a day or two


idea-community: 2017.2.2 -> 2017.2.4

Motivation for this change
I've been using 2017.2.4 for the last three days and I did not notice any problems or regressions.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] Linux

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

I decided to merge #29591 instead. Thank you :)