Unable to query ai-embed-qa-4; uninformative error

I am trying to experiment with different embedding models in a RAG application building off of the example here. It works fine when I create an NVIDIAEmbeddings object with model="nvolveqa_40k" but with model="ai-embed-qa-4" it fails at the vectorestore creation step ie
vectorstore = FAISS.from_documents(documents, document_embedder)

with the following uninformative error:
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[6], line 7
      5 # document_embedder = NVIDIAEmbeddings(model="nvolveqa_40k", model_type="passage")
      6 document_embedder = NVIDIAEmbeddings(model="ai-embed-qa-4")
----> 7 vectorstore = FAISS.from_documents(documents, document_embedder)
      8 retriever = vectorstore.as_retriever()
     10 user_input = "How do I query NVIDIA models in LangChain?"

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_core/vectorstores.py:550](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_core/vectorstores.py#line=549), in VectorStore.from_documents(cls, documents, embedding, **kwargs)
    548 texts = [d.page_content for d in documents]
    549 metadatas = [d.metadata for d in documents]
--> 550 return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:930](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py#line=929), in FAISS.from_texts(cls, texts, embedding, metadatas, ids, **kwargs)
    903 @classmethod
    904 def from_texts(
    905     cls,
   (...)
    910     **kwargs: Any,
    911 ) -> FAISS:
    912     """Construct FAISS wrapper from raw documents.
    913
    914     This is a user friendly interface that:
   (...)
    928             faiss = FAISS.from_texts(texts, embeddings)
    929     """
--> 930     embeddings = embedding.embed_documents(texts)
    931     return cls.__from(
    932         texts,
    933         embeddings,
   (...)
    937         **kwargs,
    938     )

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/embeddings.py:142](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/embeddings.py#line=141), in NVIDIAEmbeddings.embed_documents(self, texts)
    136     batch = texts[i : i + self.max_batch_size]
    137     truncated = [
    138         text[: self.max_length] if len(text) > self.max_length else text
    139         for text in batch
    140     ]
    141     all_embeddings.extend(
--> 142         self._embed(truncated, model_type=self.model_type or "passage")
    143     )
    144 return all_embeddings

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/embeddings.py:105](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/embeddings.py#line=104), in NVIDIAEmbeddings._embed(self, texts, model_type)
    102     if self.truncate:
    103         payload["truncate"] = self.truncate
--> 105 response = self.client.get_req(
    106     model_name=self.model,
    107     payload=payload,
    108     endpoint="infer",
    109 )
    110 response.raise_for_status()
    111 result = response.json()

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py:392](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py#line=391), in NVEModel.get_req(self, model_name, payload, invoke_url, stop, endpoint)
    390 if payload.get("stream", False) is True:
    391     payload = {**payload, "stream": False}
--> 392 response, session = self._post(invoke_url, payload)
    393 return self._wait(response, session)

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py:220](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py#line=219), in NVEModel._post(self, invoke_url, payload)
    218 session = self.get_session_fn()
    219 self.last_response = response = session.post(**self.last_inputs)
--> 220 self._try_raise(response)
    221 return response, session

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py:303](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py#line=302), in NVEModel._try_raise(self, response)
    301     body += "\nPlease check or regenerate your API key."
    302 # todo: raise as an HTTPError
--> 303 raise Exception(f"{header}\n{body}") from None

Exception: [400] Bad Request
Inference error
RequestID: e7d48fb8-0a64-49d3-8cf3-a0c9ddbddbb4

I had noticed that for generation models this error sometimes simply means that a newer package version is required and I have filed an issue requesting for more informative errors in that case but with this model, I get this error even with the latest version (0.0.9) and a newly generated API key.
If the model is not yet supported, can it be hidden from the output of available_models?

If the model is not yet supported, can it be hidden from the output of available_models?

this is a good suggestion, also mentioned in https://github.com/langchain-ai/langchain-nvidia/issues/26 for chat completion models

@aishwaryap is it possible one of the documents you are sending to FAISS is empty? the service rejects empty content.

@mattf I'm reasonably sure they are not. I took a working example with the nvolveqa_40k model and replaced just the embedding model with ai-embed-qa-4 to get this issue. I assume empty documents would error out with any model.
That said I can create a self complete example and add it for testing.

Sample self contained script:
import os
from langchain_community.document_loaders import AsyncHtmlLoader
from langchain_community.document_transformers import Html2TextTransformer
from langchain.text_splitter import CharacterTextSplitter
from langchain.vectorstores import FAISS
from langchain_nvidia_ai_endpoints import NVIDIAEmbeddings

os.environ["NVIDIA_API_KEY"] = "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"

urls = ["https://python.langchain.com/docs/integrations/chat/nvidia_ai_endpoints/"]
loader = AsyncHtmlLoader(urls)
docs = loader.load()
html2text = Html2TextTransformer()
docs_transformed = html2text.transform_documents(docs)
text_splitter = CharacterTextSplitter(chunk_size=2000, chunk_overlap=200)
chunks = text_splitter.split_documents(docs_transformed)
print("Number of chunks: ", len(chunks))
print("----------------------")
print("Chunk text at index 2: \n")
print(chunks[1].page_content)
print("----------------------")
nvolveqa_embedder = NVIDIAEmbeddings(model="nvolveqa_40k")
nvolveqa_vectorstore = FAISS.from_documents(chunks, nvolveqa_embedder)
nvolveqa_retriever = nvolveqa_vectorstore.as_retriever()
user_input = "How do I query NVIDIA models in LangChain?"
nvolveqa_retrieved = nvolveqa_retriever.get_relevant_documents(user_input)
print("----------------------")
print("Top retrieved chunk text from nvolveqa-40k: \n")
print(nvolveqa_retrieved[0].page_content)
print("----------------------")
embedqa4_embedder = NVIDIAEmbeddings(model="ai-embed-qa-4")
embedqa4_vectorstore = FAISS.from_documents(chunks, embedqa4_embedder)
embedqa4_retriever = embedqa4_vectorstore.as_retriever()
user_input = "How do I query NVIDIA models in LangChain?"
embedqa4_retrieved = embedqa4_retriever.get_relevant_documents(user_input)
print("----------------------")
print("Top retrieved chunk text from ai-embed-qa-4: \n")
print(embedqa4_retrieved[0].page_content)
print("----------------------")

My output (stderr + stdout):
Fetching pages: 100%|###########################################################################################| 1[/1](http://localhost:8888/1) [00:00<00:00, 27.39it[/s](http://localhost:8888/s)]
Created a chunk of size 2313, which is longer than the specified 2000
Number of chunks:  22
----------------------
Chunk text at index 2:

* Graphs

    * Callbacks

    * Chat loaders

    * Adapters

    * Stores

  *   * Components
  * Chat models
  * NVIDIA AI Foundation Endpoints

On this page

# NVIDIA AI Foundation Endpoints

The `ChatNVIDIA` class is a LangChain chat model that connects to NVIDIA AI
Foundation Endpoints.

> NVIDIA AI Foundation Endpoints give users easy access to NVIDIA hosted API
> endpoints for NVIDIA AI Foundation Models like Mixtral 8x7B, Llama 2, Stable
> Diffusion, etc. These models, hosted on the NVIDIA NGC catalog, are
> optimized, tested, and hosted on the NVIDIA AI platform, making them fast
> and easy to evaluate, further customize, and seamlessly run at peak
> performance on any accelerated stack.
>
> With NVIDIA AI Foundation Endpoints, you can get quick results from a fully
> accelerated stack running on NVIDIA DGX Cloud. Once customized, these models
> can be deployed anywhere with enterprise-grade security, stability, and
> support using NVIDIA AI Enterprise.
>
> These models can be easily accessed via the `langchain-nvidia-ai-endpoints`
> package, as shown below.

This example goes over how to use LangChain to interact with and develop LLM-
powered systems using the publicly-accessible AI Foundation endpoints.

## Installation

    %pip install --upgrade --quiet langchain-nvidia-ai-endpoints

    Note: you may need to restart the kernel to use updated packages.

## Setup

**To get started:**

  1. Create a free account with the NVIDIA NGC service, which hosts AI solution catalogs, containers, models, etc.

  2. Navigate to `Catalog > AI Foundation Models > (Model with API endpoint)`.

  3. Select the `API` option and click `Generate Key`.

  4. Save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints.
----------------------
----------------------
Top retrieved chunk text from nvolveqa-40k:

* Graphs

    * Callbacks

    * Chat loaders

    * Adapters

    * Stores

  *   * Components
  * Chat models
  * NVIDIA AI Foundation Endpoints

On this page

# NVIDIA AI Foundation Endpoints

The `ChatNVIDIA` class is a LangChain chat model that connects to NVIDIA AI
Foundation Endpoints.

> NVIDIA AI Foundation Endpoints give users easy access to NVIDIA hosted API
> endpoints for NVIDIA AI Foundation Models like Mixtral 8x7B, Llama 2, Stable
> Diffusion, etc. These models, hosted on the NVIDIA NGC catalog, are
> optimized, tested, and hosted on the NVIDIA AI platform, making them fast
> and easy to evaluate, further customize, and seamlessly run at peak
> performance on any accelerated stack.
>
> With NVIDIA AI Foundation Endpoints, you can get quick results from a fully
> accelerated stack running on NVIDIA DGX Cloud. Once customized, these models
> can be deployed anywhere with enterprise-grade security, stability, and
> support using NVIDIA AI Enterprise.
>
> These models can be easily accessed via the `langchain-nvidia-ai-endpoints`
> package, as shown below.

This example goes over how to use LangChain to interact with and develop LLM-
powered systems using the publicly-accessible AI Foundation endpoints.

## Installation

    %pip install --upgrade --quiet langchain-nvidia-ai-endpoints

    Note: you may need to restart the kernel to use updated packages.

## Setup

**To get started:**

  1. Create a free account with the NVIDIA NGC service, which hosts AI solution catalogs, containers, models, etc.

  2. Navigate to `Catalog > AI Foundation Models > (Model with API endpoint)`.

  3. Select the `API` option and click `Generate Key`.

  4. Save the generated key as `NVIDIA_API_KEY`. From there, you should have access to the endpoints.
----------------------
---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[1], line 32
     30 print("----------------------")
     31 embedqa4_embedder = NVIDIAEmbeddings(model="ai-embed-qa-4")
---> 32 embedqa4_vectorstore = FAISS.from_documents(chunks, embedqa4_embedder)
     33 embedqa4_retriever = embedqa4_vectorstore.as_retriever()
     34 user_input = "How do I query NVIDIA models in LangChain?"

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_core/vectorstores.py:550](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_core/vectorstores.py#line=549), in VectorStore.from_documents(cls, documents, embedding, **kwargs)
    548 texts = [d.page_content for d in documents]
    549 metadatas = [d.metadata for d in documents]
--> 550 return cls.from_texts(texts, embedding, metadatas=metadatas, **kwargs)

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py:930](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_community/vectorstores/faiss.py#line=929), in FAISS.from_texts(cls, texts, embedding, metadatas, ids, **kwargs)
    903 @classmethod
    904 def from_texts(
    905     cls,
   (...)
    910     **kwargs: Any,
    911 ) -> FAISS:
    912     """Construct FAISS wrapper from raw documents.
    913
    914     This is a user friendly interface that:
   (...)
    928             faiss = FAISS.from_texts(texts, embeddings)
    929     """
--> 930     embeddings = embedding.embed_documents(texts)
    931     return cls.__from(
    932         texts,
    933         embeddings,
   (...)
    937         **kwargs,
    938     )

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/embeddings.py:142](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/embeddings.py#line=141), in NVIDIAEmbeddings.embed_documents(self, texts)
    136     batch = texts[i : i + self.max_batch_size]
    137     truncated = [
    138         text[: self.max_length] if len(text) > self.max_length else text
    139         for text in batch
    140     ]
    141     all_embeddings.extend(
--> 142         self._embed(truncated, model_type=self.model_type or "passage")
    143     )
    144 return all_embeddings

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/embeddings.py:105](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/embeddings.py#line=104), in NVIDIAEmbeddings._embed(self, texts, model_type)
    102     if self.truncate:
    103         payload["truncate"] = self.truncate
--> 105 response = self.client.get_req(
    106     model_name=self.model,
    107     payload=payload,
    108     endpoint="infer",
    109 )
    110 response.raise_for_status()
    111 result = response.json()

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py:392](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py#line=391), in NVEModel.get_req(self, model_name, payload, invoke_url, stop, endpoint)
    390 if payload.get("stream", False) is True:
    391     payload = {**payload, "stream": False}
--> 392 response, session = self._post(invoke_url, payload)
    393 return self._wait(response, session)

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py:220](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py#line=219), in NVEModel._post(self, invoke_url, payload)
    218 session = self.get_session_fn()
    219 self.last_response = response = session.post(**self.last_inputs)
--> 220 self._try_raise(response)
    221 return response, session

File [~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py:303](http://localhost:8888/lab/tree/~/Documents/venvs/nvaif_env/lib/python3.12/site-packages/langchain_nvidia_ai_endpoints/_common.py#line=302), in NVEModel._try_raise(self, response)
    301     body += "\nPlease check or regenerate your API key."
    302 # todo: raise as an HTTPError
--> 303 raise Exception(f"{header}\n{body}") from None

Exception: [400] Bad Request
Inference error
RequestID: 11a69c13-e43c-4f0e-960d-98f4a3f4f706

Also verified using pip show that I am on version 0.0.9 (langchain-nvidia-ai-endpoints does not have a __version__ attribute to verify in code (issue))
(nvaif_env) ➜  ~ pip show langchain-nvidia-ai-endpoints
Name: langchain-nvidia-ai-endpoints
Version: 0.0.9
Summary: An integration package connecting NVIDIA AI Endpoints and LangChain
Home-page: https://github.com/langchain-ai/langchain
Author:
Author-email:
License: MIT
Location: /Users/apadmakumar/Documents/venvs/nvaif_env/lib/python3.12/site-packages
Requires: aiohttp, langchain-core, pillow
Required-by:

@aishwaryap thank you for the reproducer, it helped me narrow this down.
i believe the issue is some of the inputs are longer than the embedding model allows. in this case you can pass truncate="END", e.g. NVIDIAEmbeddings(model="ai-embed-qa-4", truncate="END")
this is not an issue w/ the nvolveqa_40k model because it would silently truncate your input, while the new models reject the input by default.
does that resolve your issue?

Hi @mattf, I just found this thread and I just wanted to say that your suggestion worked... at least for the issue at
vectorstore = FAISS.from_documents(documents, document_embedder)
so thanks for that information. However, I wanted to ask a follow-up question since a new Error arises when working with a chain and the vectorstore retriever. I'm trying to ingest scientific articles in PDF format and after passing the following chain:
retriever = vectorstore.as_retriever()
prompt = ChatPromptTemplate.from_messages(
        [
            (
                "system",
                "Answer solely based on the following context:\n<Documents>\n{context}\n</Documents>",
            ),
            ("user", "{question}"),
        ]
    )

chain = (
        {"context": retriever, "question": RunnablePassthrough()}
        | prompt
        | llm
        | StrOutputParser()
    )
response = chain.invoke( "A question for the PDF")

I get the following error:
 .venv\Lib\site-packages\langchain_nvidia_ai_endpoints_common.py", line 311, in _try_raise
raise Exception(f"{header}\n{body}") from None
Exception: [500] Internal Server Error
Input value error: prompt is [[5201]] long while only 2048 is supported

Any ideas how to solve this part? Thanks in advance for any help you may provide!

@apolo74 please open this as a new issue, it appears unrelated to embedding and has an informative error

Hi again @mattf, a couple of minutes ago solved this... I was using a small model (microsoft/phi-3-mini-4k-instruct). There were no more errors the moment I switch to larger models. So the error was related to the size of the LLM.
Thanks for your help!

@aishwaryap recent changes server-side should have fully resolved this. please reopen this if you still have an issue.


fix(config): correct entry for Zooz ZSE11 Param 12

Fixes an issue raised by Zooz via email.

@zwave-js-bot rename commit


Multi instance reporting

What?
We have retrieved the instanceId out of the push notification data
...
Why?
We need to accurately report the instanceId when we support multiple per app
...

CC @pusher/mobile

You can also run SwiftLint to format the code before submitting PR.
Rules from this file will be applied when formatting.
Good job 👍


[ENH] AutoETS prediction intervals simplification

This PR simplifies the probabilistic prediction functionality in AutoETS.
Previously, _predict_quantiles was implemented, but AutoETS returns intervals. Hence _predict_quantiles was removed and replaced by a more direct interface through _predict_interval.
By discussion below, the interface was previously correct, most probably, so this is merely a simplification of the logic.

Hi @fkiraly , I tested these changes with the fix branch, but I still see a difference in the results

Actually, there seems to be some randomness to this process.  A second and third run of the same code gives different results:

@ngupta23, I was already going crazy, all seemed fine...
It does not seem possible to set a random state in there, does it?

@fkiraly there is no option to set any seed. Also I think the point forecasts are reproducible. The randomness is in the intervals and quantiles.

So, all looks plausible then!
Since the logic in the conversion intervals/quantiles is identical (per inheritance) with that used in another case where we know it works (ARIMA et al), this is actually, most likely, not-a-bug.
Hence, this PR is a clean-up and not a bugfix.

@fkiraly - Here is another example. Asking for the same interval multiple times results in different results. Could it be that the prediction is changing some internal state in the model?
Why do you think this is not a bug? Should it not be reproducible?
print(forecaster.predict_quantiles(alpha=[0.1, 0.1, 0.1, 0.5, 0.5, 0.9, 0.9, 0.9]))

Why do you think this is not a bug? Should it not be reproducible?

Only for a fixed random state, which this model does not have.
Many models are non-deterministic, e.g., as they do some random sampling inside, especially when it comes to estimating variation. Having some intrinsic randomness there is not unusual and expected.
The fluctuations seem large for this particular model, and this may go together with sub-optimal performance in predicting uncertainty. But, I currently see no reason to believe that it would be "wrong" in the sense of correctly interfacing whatever the original AutoETS does.

Many models are non-deterministic, e.g., as they do some random sampling inside, especially when it comes to estimating variation. Having some intrinsic randomness there is not unusual and expected.

@fkiraly You are right, it looks like in this case, the intervals are based on simulations and those simulations are not exact or repeatable. This model has multiplicative components and hence exact method is not available
https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html
if method is None:
    exact_available = ["ANN", "AAN", "AAdN", "ANA", "AAA", "AAdA"]
    if results.model.short_name in exact_available:
        method = "exact"
    else:
        method = "simulated"
self.method = method

However, they can be made repeatable by passing a random_state inside the sktime ets.py _predict_interval method. What do you think - maybe open an enhancement request for this?
def _predict_interval(self, fh, X=None, coverage=None):
    valid_indices = fh.to_absolute(self.cutoff).to_pandas()

   start, end = valid_indices[[0, -1]]
   # Maybe add random_state in get_predictions (and accept random_state in the AutoETS initialization)?
   # https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html
   prediction_results = self._fitted_forecaster.get_prediction(
       start=start, end=end
   )

@fkiraly You are right, it looks like in this case, the intervals are based on simulations and those simulations are not exact or repeatable. This model has multiplicative components and hence exact method is not available

I also looked into it - it seems like the simulate method has a random_state that can be set?
But it does not seem like it can be easily called from the model prediction?

But it does not seem like it can be easily called from the model prediction

I think the simulate kwargs can be passed to get_prediction method. So should be possible - right?
https://www.statsmodels.org/stable/_modules/statsmodels/tsa/exponential_smoothing/ets.html


Talking in chat while having three blank spaces as your name disconnects you from the server

thanks, will check this out

This is no longer a bug, however it is a bug that you can name yourself three spaces

names self ...


Bug: TypeError: $isGridSelection is not a function

Lexical version:
0.12.5
I have tried to extract the official lexical playground package from lexical mono repo and make it an independent package, with published lexical packages as dependencies. I intend to build a custom editor based on it.
I got this error:
TypeError: $isGridSelection is not a function
The link to the repo is:
https://github.com/fyepi/lexical-playground

I use node v18.17.0, pnpm 8.6.9 and vite 5.0.0.

Duplicate of https://github.com/facebook/lexical/issues/5411

I believe this is now fixed as 0.12.6, could you confirm?

Yes.It has been fixed. Thanks @ivailop7


Comparison of both methods (Siamese NN and Search_around_sky)

Evaluate performance of the two methods and choose the best.
Generate a monitoring plot which aggregate the performance of both methods .The plot show the evolution of the both performance method according to the night difference between the alerts.
Generate somes other plots which shows the algorithm performance (time execution, memory consumption, ...) of both method.

Not implemented due to the idea of machine learning that have been aborted


Error

Hello,
I'm very interesting about Moto.Net, but it's not working on my computer.
Could you help me ?
I've paste TRBOnet.Server.exe (v5) in the MotoMond folder.
When I run MotoMond.exe, I've that :
https://forum.tetrahub.net/ext/dmzx/imageupload/img-files/51/e1673ea/c7349c5/212a2915f634f2a990c2e570d55ac4dc.jpg
Do you down why ?
Thank you

Can you send me the dump file?

https://support.kaspersky.com/common/diagnostics/12401

Hello,
Unfortunetly, the dump file is not created because of an error :
[image: dump file.jpg]
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
Sans
virus.www.avast.com
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Le dim. 14 mai 2023 à 23:37, Patrick Boyd @.***> a
écrit :

https://support.kaspersky.com/common/diagnostics/12401
—
Reply to this email directly, view it on GitHub
https://github.com/pboyd04/Moto.Net/issues/9#issuecomment-1547006970,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFTFHHNWBT7EL3B6QQE7IZTXGFGALANCNFSM6AAAAAAX6VZZME
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Malo

Good evening,
I managed to create the dump files on another machine. here they are.
conhost.DMP
https://drive.google.com/file/d/10xp7FGyQWhQIRHWnMja_3wtJOCk40Hmf/view?usp=drive_web
MotoMond.DMP
https://drive.google.com/file/d/1uvpxtqYgxAgbvOOJdOwZoxatbCY_8Xl9/view?usp=drive_web
WerFault.DMP
https://drive.google.com/file/d/19TAnB7ub01xc3rLdPb48ZiZmv0AfCe41/view?usp=drive_web
Le lun. 15 mai 2023 à 13:18, Malo @.***> a écrit :

Hello,
Unfortunetly, the dump file is not created because of an error :
[image: dump file.jpg]
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Sans
virus.www.avast.com
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#m_7626657878514774920_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Le dim. 14 mai 2023 à 23:37, Patrick Boyd @.***> a
écrit :

https://support.kaspersky.com/common/diagnostics/12401
—
Reply to this email directly, view it on GitHub
https://github.com/pboyd04/Moto.Net/issues/9#issuecomment-1547006970,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFTFHHNWBT7EL3B6QQE7IZTXGFGALANCNFSM6AAAAAAX6VZZME
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Malo

--
Malo

Hello,
What do you think about these dump file please ?
Le jeu. 1 juin 2023 à 22:39, Malo @.***> a écrit :

Good evening,
I managed to create the dump files on another machine. here they are.
conhost.DMP
https://drive.google.com/file/d/10xp7FGyQWhQIRHWnMja_3wtJOCk40Hmf/view?usp=drive_web
MotoMond.DMP
https://drive.google.com/file/d/1uvpxtqYgxAgbvOOJdOwZoxatbCY_8Xl9/view?usp=drive_web
WerFault.DMP
https://drive.google.com/file/d/19TAnB7ub01xc3rLdPb48ZiZmv0AfCe41/view?usp=drive_web
Le lun. 15 mai 2023 à 13:18, Malo @.***> a écrit :

Hello,
Unfortunetly, the dump file is not created because of an error :
[image: dump file.jpg]
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail Sans
virus.www.avast.com
https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail
<#m_-2719641723964632546_m_7626657878514774920_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Le dim. 14 mai 2023 à 23:37, Patrick Boyd @.***> a
écrit :

https://support.kaspersky.com/common/diagnostics/12401
—
Reply to this email directly, view it on GitHub
https://github.com/pboyd04/Moto.Net/issues/9#issuecomment-1547006970,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AFTFHHNWBT7EL3B6QQE7IZTXGFGALANCNFSM6AAAAAAX6VZZME
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Malo

--
Malo

--
Malo

I'm not able to find anything in the dump file that indicates any kind of fault.


pull request for e2e test at 01/31/2017 18:56:12

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


testcase fix

Testcase testRenameZipFileAfterExtractionWithInputStreamSucceeds would fail on Windows. This is because the renameTo would return fail if the target exists, while it would work successfully on Linux/MacOS even if the target exists.
Therefore the newFile need to be deleted on Windows, otherwise the generatedZipFile.renameTo(newFile) will return false and this testcase would fail.
for reference:
javadoc
file-renameto-fails

Thanks for fixing it


[FEATURE] gRPC Health Check must be implemented in case flagd is accessed via AWS ALB

Requirements
To configure flagd to be accessed via AWS ALB when deployed as standalone rather than as OpenFeature Operator, it must be specified the health check related properties of ALB, which must be implemented as '/AWS.ALB/healthcheck' path in flagd.

@Kavindu-Dodan @bacherfl @thisthat is this as simple as making healthz/readyz configurable?

@kinyoklion  I don't fully understand your requirement.
Regardless of the application you are deploying (here you are referencing to flagd), AWS allows your to configure health check endpoint of your ALB.
Please check these steps to configure this ^1 in your EC2 instance.
flagd expose readiness & liveness probes as described in our documentation ^2. So you can configure health check endpoints for them.

I think you at-ed @kinyoklion when you meant to at @kworkbee

@Kavindu-Dodan I think you at-ed @kinyoklion when you meant to at @kworkbee 😅

yeah, my bad 😞 . updated the comment :D

@toddbaert @Kavindu-Dodan Hello, Thanks for your answer.
I tried it, but due to AWS ALB Ingress Controller setting, if backend protocol version uses gRPC, it seems that only gRPC is available as healthcheck. You can specify the (Not HTTP) Status Code only as the status code of the gRPC.

@Kavindu-Dodan, would it be possible to add support for gRPC-based health checks in addition to HTTP?

I can look into this. It seems that we do indeed need a gRPC healthcheck that returns a value between 0-99 (probably 12) in order to satisfy AWS' requirements.

I think this is a legimitate issue and we should fix it by adding a standard gRPC health endpoint, which conforms to this standard proto.

@kworkbee I've opened PR to fix this. According to what I see in the answer for AWS here, they should support the standard gRPC health check (/grpc.health.v1.Health/Check), which is what I've implemented. Note, we are returning code 0 (success) on the health check.

@kworkbee this has been released https://github.com/open-feature/flagd/pkgs/container/flagd
According to my research this should work with the AWS infra, since it's a standard gRPC healthcheck. I tested it manually with the standard proto and it worked, but I never actually tested it in AWS. Let us know if this helps you.

@toddbaert, I tested with a new release and found that it works! Thanks for your support.


Update dependencies

Running glide get github.com/pkg/errors in #1011 somehow insists on updating all other dependencies also. This PR extracts the result of running glide up.
The API of https://github.com/fatih/color has changed, this PR contains only relatively hacky fixes. I am planning on a larger refactor of how we handle colors when I get a Windows machine running in order to fix #954 and #1017.

Looking at the repo, I don't believe the API for https://github.com/fatih/color changed. I think those changes can be reverted

The error message one gets with the latest version of color is:
color.New(color.Bold).Add(color.FgCyan).Sprintf undefined (type *color.Color has no field or method Sprintf)

The  Sprintf method is no longer documented. The new behavior of using SprintFunc() for inserting colors into strings is documented at https://github.com/fatih/color#insert-into-noncolor-strings-sprintfunc.

Correction, running glide  update somehow switched to a version of color from 2016. I have hard-coded the latest version of color in glide.lock and now it works without the changes. Really strange, seems like a bug in glide.

you removed the test imports from the glide.yaml and glide.lock but it appears they still appear in the vendor folder.

I don't think we can vendor ginkgo as we need its executable

After experimenting with a few ideas like compiling the binaries in vendored dependencies using https://github.com/roboll/go-vendorinstall (which doesn't work), this is where I landed:

all dependencies for which we need binaries are installed as before, using go get. This leaves us vulnerable for random breaking changes on their respective master branches at any time, but I don't see a way to deal with it better right now. At least gometalinter has some hacky form of versioning using their Git URL, however well that works.
all other test dependencies (i.e. the ones we use as libraries in tests) are now versioned via Glide in the testImports section and are removed from the setup script. They are determined automatically by glide and are version locked. At least we get a bit of stability that way. They are updated explicitly via glide up and we can test those updates.


Dev

Dev ended up with a module version behind master somehow.  Fixing that now.

@TerrapinStation PSGallery has been updated. For the latest:
Find-Module PSGitLab | install-module -force

Nice! Thx @ngetchell
Sent from my iPhone

On Aug 15, 2016, at 6:10 PM, Nicholas M. Getchell<EMAIL_ADDRESS>wrote:
@TerrapinStation PSGallery has been updated. For the latest:
Find-Module PSGitLab | install-module -force
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.


Code block disappears when resizing chat sidebar

VS Code Version:
OS Version:
Logs:

Steps to Reproduce:

Please give some more info, a screen recording might help.
You can also check for errors in the "Window" output channel

Screen Recording 2024-02-27 at 19.34.39.mov
https://drive.google.com/file/d/1nd2XkuIu_ExJUQNV5nuMlYmPMvI8zk_b/view?usp=drive_web
Please find attached a screen recording
Thanks and best Regards
Lamat
Fishtankproduction
Email   @.***
Website   https://fishtankproduction.com
On Tue, Feb 27, 2024 at 7:03 PM Rob Lourens @.***>
wrote:

Please give some more info, a screen recording might help.
You can also check for errors in the "Window" output channel
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/vscode-copilot-release/issues/961#issuecomment-1966562775,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWEODQVSMDBT3SDMBWGTTF3YVXODNAVCNFSM6AAAAABD3XUEUCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNRWGU3DENZXGU
.
You are receiving this because you authored the thread.Message ID:
@.***>

Attachments can't be sent over email, you will have to upload it to github.com


Add MustGetGlobal() for more convenient global class usage

Add MustGetGlobal() for more convenient global class usage.
For example:
package foo

import "github.com/hack-pad/safejs"

var jsUint8Array = safejs.MustGetGlobal("Uint8Array")

func Doit(b []byte) (safejs.Value, error) {
    arr, err := jsUint8Array.New(len(b))
    if err != nil {
        return safejs.Value{}, err
    }
    _, err = safejs.CopyBytesToJS(arr, b)
    return arr, err
}

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

14 of 18 (77.78%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.8%) to 91.148%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

global.go
14
18
77.78%

Totals

Change from base Build<PHONE_NUMBER>:
-0.8%

Covered Lines:
278

Relevant Lines:
305

💛  - Coveralls


Floating table of contents - Bullets

Consider removing bullets from the floating table of contents. Bullets are not used in the floating table of contents in the pages of other service providers documentation (e.g. Google and GitHub). Indentation appears to be the preferred method, which I personally find less distracting then indentation accompanied by bullets.

Fixed by https://github.com/PASTAplus/web-x/commit/7f6dfe8497c213cc70bc54e5a44d3b2a0a693fab


Implement new worker semantics - Part II

Purpose

Add support for multiple receive up to desugar
Allow conditional send actions and implement channel auto-close invocations.

Approach
N/A
Samples
N/A
Remarks
N/A
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

Closing as all the commits are included in #41797


Visualize Plotly in viser

Plotly in viser! Could be useful for visualizing images/plots, and interacting with them, as opposed to markdown.
The current pipeline is:

From python side, create Plotly figure
send the json string to client side (https://plotly.com/python-api-reference/generated/plotly.io.to_json.html#plotly.io.to_json)
Render Plotly figure using Plotly react (https://github.com/plotly/react-plotly.js)

, then the python code can re-send the figure upon modification.
https://github.com/nerfstudio-project/viser/assets/10284938/b9582633-081d-4bf2-8a48-aa68f303f1e1

[x] Use Plotly.py's js-dist-min instead, to avoid Plotly.py / js version mismatches.
[ ] Match plotly theming with viser.
[ ] Make simpler interface for "simpler" plots (e.g., lineplots).
[ ] Easy option for plotly dropdown?
[x] Refine the "plot expand" functionality using modals, seen in end of video.
[ ] remove go.plot typing from _gui_api.py

[ ] Test latency e.g., via using images.
[ ] Check size of JSON string / state being sent over to the client.

Per brent:

[ ] Make the border and the text style similar to the folder style.
[ ] Icon -- make the "expand" icon next to the label (ie the plot name)?

Current vis for the plot (see below):

Remove plot label (this can be added in the plotly plot itself).
Remove "expand" icon, via clicking the plot. Hint appears upon hover.

https://github.com/nerfstudio-project/viser/assets/10284938/601a4d87-5e67-419a-9519-b64679dd0597

Also, making plots dropdown can be easily done with gui_add_dropdown, without dealing with plotly directly (https://plotly.com/python/dropdowns/).
https://github.com/nerfstudio-project/viser/assets/10284938/f1db069f-11ba-46cf-8b5c-e0feaaa39b4e


Suggestion: Add option to disable everything but displaying the tags

I really like the way the extension displays the tags for every cell.
For my application, however, I would like to disable all the remaining features - in particular the buttons for moving cells up/down.
Would it be possible to add an option in the settings to achieve this?

Thanks for reaching out @Phoemuex
Yes it should be easy to achieve by filtering the POSITIONED_BUTTONS list with the one specified by a setting or to skip that statement with a boolean setting:
https://github.com/jupyterlab-contrib/jlab-enhanced-cell-toolbar/blob/0853ad57c579cff25442cc2189688dbb77acb76b/src/celltoolbartracker.ts#L147
PRs are welcomed if you want to give it a try.


#743 add vite react

realted to #743

POST /api/projects/init
POST /api/vite/react
npm i
npm run dev

Could you add a new config here:

new json here: https://github.com/jhipster/jhipster-lite/tree/main/tests-ci/config
new config in https://github.com/jhipster/jhipster-lite/blob/main/tests-ci/generate.sh
and new config in github actions: :https://github.com/jhipster/jhipster-lite/blob/main/.github/workflows/github-actions.yml#L96-L101

So we are sure Vite/React is tested in CI

I add it in generate.sh but i don't know how to do in the 2 others config, can i have an exemple ?

I add it in generate.sh but i don't know how to do in the 2 others config, can i have an exemple ?

1st part

duplicate one file in https://github.com/jhipster/jhipster-lite/tree/main/tests-ci/config and edit it
be careful on the path value, and the name of the json file (should be similar)

2nd part

add the new entry here: https://github.com/jhipster/jhipster-lite/blob/main/.github/workflows/github-actions.yml#L96-L101

1st part

duplicate one file in https://github.com/jhipster/jhipster-lite/tree/main/tests-ci/config and edit it
be careful on the path value, and the name of the json file (should be similar)

2nd part

add the new entry here: https://github.com/jhipster/jhipster-lite/blob/main/.github/workflows/github-actions.yml#L96-L101

so i create a config, and i have to add this config in github-actions in generation/strategy/matrix/app ? the name of the config file must be something like tomcat-vite-react ?

so i create a config, and i have to add this config in github-actions in generation/strategy/matrix/app ? the name of the config file must be something like tomcat-vite-react ?

Yes, sounds good :)

i guess i forgot something in the config but i can't get it

Some conflict to resolve

Thanks a lot for your hard work here @flcarre
I'm preparing right now a new release of JHipster Lite v0.0.10, and I'll review this one soon. It means, it can be included in the next release v0.0.11 :)

Nice 🙂

I didn't finish to test it, but I have already 2 comments:

after building, there is a generated file at root folder. The target folder should be changed I think
need to add index.tsx to Sonar ignore, as it is not needed in coverage, like index.ts for Angular and Vue

need to add index.tsx to Sonar ignore, as it is not needed in coverage, like index.ts for Angular and Vue

I'm not sure to understand, i need to exclude it somewhere else of jest.config.js ?
  collectCoverageFrom: [     'src/main/webapp/**/*.{js,ts,vue}',     '!src/main/webapp/**/*.component.ts',     '!src/main/webapp/app/main.ts',     '!**/*.d.ts',   ],

The exclusion for Sonar to add is here: https://github.com/jhipster/jhipster-lite/blob/main/src/main/resources/generator/server/sonar/sonar-fullstack-project.properties.mustache#L17

Thanks and congrats for your contribution @flcarre

https://opencollective.com/generator-jhipster/expenses/64816

@flcarre : approved :)


refactor(adapter): extract image_area from image_show

Exposing image_area separately allows future Lua integration to handle image placement more flexibly, without needing to invoke full rendering. This change promotes modularity and simplifies obtaining image dimensions alone.
This refactor provides the foundation for feature PR #1897, preparing for easier integration.

Why not implement the feature directly in image_show() so that it supports alignment, instead of requiring an extra step to get image_size() first?
To me, this is a significant performance loss, as now two I/O reads are necessary. Plus, for large images that haven't been downscaled, an additional downscale() operation is needed.

If modifying the image_show function signature is acceptable, I could consider submitting a new PR with these changes. I’m definitely open to other potential approaches if they better suit the codebase

Sure, I can accept changing the function signature as long as it benefits performance.

As long as the original image_show performs downscaling, any image_area extracted from it will already have this downscaling applied.

If I understand correctly, from another PR, image_show_aligned() API calls both image_area() and image_show(), and both of these functions ultimately call downscale(). The issues here are:

The image gets fully decoded twice, even if it doesn't need to be downscaled
https://github.com/sxyazi/yazi/blob/c477f68058ca3d546aca0c1533cc582e270cc3ef/yazi-adapter/src/image.rs#L44-L45
Even when image_area() has already downscaled the image, image_show() still uses the original image without utilizing the already downscaled result. This leads to images that need downscaling getting processed twice.


Fix branch inconsistency

This PR back port bug fixes from master branch back to branch v0.4 (up to release 0.4.4). The commit differences were generated via: git log --graph --pretty=format:'%Cred%h%Creset -%C(yellow)%d%Creset %s %Cgreen(%cr)%Creset' --abbrev-commit --date=relative v0.4-branch..v0.4.4
Patches to be included in v0.4.5 will be committed via another PR.
The commits which has been cherry picked:

1b47ac4 - Update Changelog (#447) (2 weeks ago) <- This was in master branch but not on release v0.4.4. After this PR is merged, will submit another PR to update the changelog so that the changes listed are consistent with the state of v0.4 branch after the merge.
b6455e2 - Bump chart version (2 weeks ago)
eefdc35 - Fix logging (#430) (4 weeks ago)
5fcc30f - Update GKE installation and chart values to work with 0.4.3 (#434) (4 weeks ago)
833d495 - Remove "resource" concept and the need to specify a kind in feature sets (#432) (4 weeks ago)
2a33f7b - Publish datatypes/java along with sdk/java (#426) (5 weeks ago)
20ce16e - Bump hibernate-validator from 6.0.13.Final to 6.1.0.Final in /ingestion (#421) (5 weeks ago)
913e7c9 - Add documentation for bigquery batch retrieval (#428) (4 weeks ago) ?
6681d4f - Update basic Feast example to Feast 0.4 (#424) (5 weeks ago)
8bdc38c - Introduce datatypes/java module for proto generation (#391) (5 weeks ago)
0e31aef - Update Changelog (#423) (5 weeks ago)

These commits will be left out:

bce068a - Add issue_label_bot.yaml (2 weeks ago)
c82ba8f - Fix null pointer exception in Dataflow Runner due to unserializable backoff (#417) (5 weeks ago) <- This commit was already present in branch v0.4 (https://github.com/gojek/feast/commit/cc884b4480c50555d8a9f28e0629d138d0d9bd36)
1afec9b - Increase resource requests to distribute tests (#418) (5 weeks ago)
1a00db1 - (README): add a link for user to view docs on GitBook (#385) (5 weeks ago)
58527de - Remove CONTRIBUTING.md (#411) (5 weeks ago)
9912d45 - GitBook: [master] one page modified (5 weeks ago)
17e7dca - Add PR template (#416) (5 weeks ago)
ba1c828 - Update protos with Tensorflow data validation schema (#438) (3 weeks ago) <- Should be in Feast v0.5
ba1c828 - Update protos with Tensorflow data validation schema (#438) (3 weeks ago) <- Should be in Feast v0.5
bb960a7 - Parameterize end to end test scripts. (#433) (4 weeks ago)
d12bcce - Fix missing CI dependency for Netlify (5 weeks ago)
6dae18f - Fix missing CI dependency in Python SDK documentation building (5 weeks ago)

/retest

/retest

/retest test-end-to-end-batch-java-8

/retest

All test for Java 8 related pass. Merging manually because the failing tests are for java 11 when the prow config has not been updated to igore those tests


Test-Output support

Added support for test-output (#51)
During development also converted to git modules (#78, #79, #80) and update libraries to use helm 2.15.

Closed this pullrequest, to create a new with additional features.


pip-sync claims ">=X" is "incompatible" with "==X" in second reqs.txt file

When syncing requirements.txt alongside another requirements file (in my case, there's runtime requirements and develop-time requirements), pip-sync forgets its math skills.
Environment Versions

OS Version: MacOS Sierra 10.12.4
Python version:  3.5.1
pip version:  9.0.1
pip-tools version: 1.9.0

Steps to replicate
Use pip-compile
Use pip-sync like so: pip-sync requirements.txt dev_requirements.txt
Expected result
Sync'ed up virtual env, ready to go.
Actual result
Incompatible requirements found: invoke>=0.15.0 (from -r requirements/dev.txt (line 12)) and invoke==0.15.0 (from -r requirements.txt (line 57))

Unclear why this is incompatible.

I looked in the cound and found this comment:
https://github.com/jazzband/pip-tools/blob/master/piptools/sync.py#L81
pip-sync was designed to be used with the output of pip-compile only. This means that both files should only have pinned requirements.
If one of the files has unpinned requirements (like invoke>=0.15.0 (from -r requirements/dev.txt (line 12))), then that file is certainly not the output of a pip-compile.
So here's why it's considered incompatible. pip-sync didn't take the math class, that's pip-compile thing. I hope that helps you out @mkoistinen.
The error message could definitely be improved though, it's quite misleading.

lol. OK. Thanks for the explanation.

Doc updated by #609, closing.


Gateway Architecture Redesign

Current Gateway architecture might not be good enough to support others than Ethereum protocols.
For gateway to be more "universal", it must have a code architecture that allows:

Easy connection to different blockchains
Have a proper interface to allow communication between Python strategies and JS protocols interactions
Allow "universal" point of connection between blockchain protocols and any program written with Python.

The end goal is to make Gateway a universal tool to be used not only with Hummingbot but work as a "translator" between blockchain protocols/smart contracts and any program based on Python.

I will write a design doc and then pass to @Nullably and @martinkou for review

@mifeng
Any updates with this? Actually I had to take a look on gateway-api in last 2 days and I see a lot of problem in code. Look like code written by Python developers.
So, if Hummingbot planned some plan about new architecture of gateway-api maybe I can see it and help in some cases?

@mifeng @martinkou
I looked inside the current api design and I have some idea about building the next implementation.
I can offer a few ways for realization of next gen gateway-api.

Standard REST API + Swagger client generation
Using GraphQL to make our API more flexible
Using gRPC

1. Standard REST API + Swagger client generation
This old school way is easy for understanding and can be used by any external/internal service.
In this case really important made API by strong standards because in other cases this can become trash. We need to keep high quality standards and keep api scalable.

In this case our users can use client SDK which will be generated from Swagger data.
I think this is possible by using any Typescript backend framework with Swagger generator. A lot of frameworks can provide this feature, TSED or NestJS as examples.
Props

Easy to understand
Auto generated client SDK based on Swagger for every language
Scalable

Old school way
Important keep REST API standards of project

2. Using GraphQL for make our API more flexible
GraphQL is gool query language for api.
GraphQL api can be really flexible and scalable, but this can require a little bit more effort for understanding and using.
We also can use the generated library for access to API from many programming languages like with REST + Swagger.
In this case we also can use many backend frameworks with Typescript.
So, that can be a good choice for gateway api, but that makes our team read more about GraphQL and about how to use GraphQL API in Python.
Props

Modern way
Easy to extends
Can return many data by one query
Strong typesation and validation out of box
A lot of tools for client SDK generation or using based on schema
No need make POST instead GET only for hide data from client =) (I see this way in code)

Difficult to start without any experience
Need make some changes in backend of bot
Not extremely fast (but very often this not depend on GraphQL but depend on developer)

Using gRPC
Good way for using gateway as microservice only, like microservice for backend of bot.
In this case it looks like the best way to start it as part of the system in docker.
So, in this way this service will be just one more component of the backend for the bot.
Because we use gRPC this should be easy to extends and safety. Communication between bot and gateway-api will fast. But in this case we need to implement support of this protocol for bot and nodejs application.
I'm sure what we also can and should use Typescript for this service.
Good way for using this microservice only like microservice for backend of bot.
Props

Safety
Good way for makes communications between parts of system

Difficult to start without any experience
Need make some changes in backend of bot
Not easy find community support

Final thoughts
So, actually I see a way with GraphQL more profitable, but in case if nobody in the team wanna use it I think the best way will be standard REST API.
We can have additional call with yours for speaking about that.

After looking at the current design, I realized that you need to develop a standardized api for exchange transactions across decentralized exchanges.
So, looks like REST API that what you need.
I on vacation, so, I started working on that.

Nice points @james-hummingbot, @Nullably, @phbrgnomo
Actually I already implemented some part of new version gateway-api.
Pls, take a look and I ready for make discuss about that.
https://github.com/evseevnn/hummingbot-gateway-api

Thanks @evseevnn for sharing your repo. I think there are a lot of good ideas in there on how to structure gateway-api better and in a more type safe way. Let me think a bit more about how to incorporate this into our repo.

@phbrgnomo actually I planned reimplement current api and then we can just replace current implementation without changing anythings on Hummingbot side. Then next step will working on protocol of connection to gateway and another refactoring and improvments.
I think this is good start for our redesign task.

@james-hummingbot ok, I will then close this issue and move the discussion to the related Epic (#44)


"python3 -m display_server.py" causes over %100 CPU use

"python3 -m display_server.py" causes over %100 CPU use when there is no HDMI or OLED display attached. it depletes battery REALLY FAST!

You can stop the service...

List the services running sudo systemctl -l --type==service to get the service name

It probably will fall under:

jetcard_display
jetcard_display.service

Stop or disable the service

sudo systemctl stop <service name>
sudo systemctl disable <service name>


Update Glossary of UserGuide.md

Include the term JAR under the Glossary part.

Add a new term.
Reasoning: The term "JAR" is used indiscriminately in the User Guide


[question] why stop_gradient on action taken w.r.t. categorical distribution but not other distribution

In common/distribution, when computing neglogp, stop_gradient only applies to action taken w.r.t. categorical distribution but not other distribution. Shouldn't stop_gradient always be applied to the sampled actions as they are functions of network parameter? I am thinking about this in the context of implementing PPO2, where actions are sampled from the 'old policy'.

Hello,
the stop_gradient is applied here because of a change in Tensorflow api (see doc).
In practice, because the sampled actions are originally numpy arrays (the one from the old policy), we could remove it.
Those actions are not connected to the graph and no gradient will be backpropagated (only gradient for the current actions self.logits will be computed).
That's why it is not done for others distributions.

Got it, thanks!
Another question: when using PPO2,  I use minibatch=1 in training and somehow the probability distribution never improve. Could you know any potential issue with minibatch=1? Thanks again!


How to use this Plugin directly in a C++ Class?

Hello @videofeedback
I have a question about this Plugin. Is it possible to use this Plugin directly in a C++ class? If so, Can you give me some hints? I want this Plugin to read data from Arduino Due over Serial Port.
I appreciate any help you can provide.
Regards
Jemish

You can. Look at the C++ calls from this folder:
https://github.com/videofeedback/Unreal_Engine_SerialCOM_Plugin/tree/main/SerialCOM/Source/SERIALCOM


Add BLENDMODE.MULTIPLY as an option for particle blending

Feature request
Add another blendmode to allow particles to multiply against each other. This should work with alpha similar to BLENDMODE_STANDARD and BLENDMODE_ADD, but using a multiply function instead.
This would be useful for smoke particles so that we can use a lighter color on the individual particle and have the smoke get darker with more particle density just as ADD does for brightness.

I've just pushed a new blend mode (BLENDMODE_MULTIPLY) which will multiply particle's color with current color.
Let me know if this is what is expected :)

Looks like we aren't respecting alpha with multiply like we do with BLENDMODE_ADD

Well this is expected as I've used a srcColordestColor
Would you expect to be srcAlphasrcColor*destColor ?

After checking more into it, webgl as no way to do a triple multiplication.
So if we want to use multiply, the alpha cannot be taken in account. Perhaps setting the texture background to white should be enough?

To give more context, this is how we can control the blending pipeline:
https://developer.mozilla.org/en-US/docs/Web/API/WebGLRenderingContext/blendFunc#Constants

In thinking about how we can get around this in the texture, I can't use white for the background because we are using a white texture with alpha to multiply the particle color against so we have the most flexibility with color over time in the system. We would need to multiply the particle color by the texture then add 1 minus alpha to the texture before multiplying SrcDst


Tick Counter

Can we get a tick counter for theatre of blood? like some other clients have. i feel like im missing out but dont wanne not use RuneLite.....

Can you provide some more information? What would a tick counter look like?

Alright, what i meaning is that the dps counters are based of RNG.
i would like to see a tick counter or afk timer.
meaning to be able to see if somebody (prefered team wide) is standing afk in an room or running around not doing dmg

This has been added to the pluginhub now.

Thank you very very much!!!


Does not show list

All i get is:
HI, (name)
Active Subscription found
but no list of completed tracks or anthing else and i've tried different arguments and cannot get a download started.

The tool only lists your completed courses.

I'm doing a career track, however it consists of multiple courses. Some of said courses, I've already completed.
I can find those courses on https://learn.datacamp.com/courses , e.g. https://learn.datacamp.com/courses/intermediate-python
However, the output of your tool is empty for the following:
> python downloader.py -t <SECRET_TOKEN> -l LIST
Hi, Maximilian
Active subscription found
====================================================================================================

Better to say, the tool does not finish. I can only kill it via CTRL+C.
Also, this:
> python downloader.py -t <SECRET_TOKEN> -l T -a -p dump
Hi, Maximilian
Active subscription found
====================================================================================================
downloader.py:107: DeprecationWarning: isAlive() is deprecated, use is_alive() instead
  while thread.isAlive():
 No completed tracks found!

I can provide my token if it helps.

See pull request:

https://github.com/TRoboto/datacamp-downloader/pull/17 "Update README.md" to avoid confusion when reading the call-snippets in the README
https://github.com/TRoboto/datacamp-downloader/pull/18 "Update downloader.py" to avoid confusion when -l/--list is called with anything other than T or C

Thank you for your contribution. Your pull requests have been merged successfully.


language package management tooling missing hashes

I'm sorry for the very general name of this issue, if anyone can come up with a better title for this problem please make a suggestion :-)
I am looking for the best solution for the following problem that I run into quite a lot:
The application I am trying to package has dependencies from it's language's packaging system, that is not tracked in nixpkgs (npm, rubygems, maven, rust crates, ...). There is tooling to adapt the dependency definitions from the language's package management system to nix (yarn2nix, bundix, ..), but since the original dependency definitions don't contain usable hashes for all dependencies, or are missing the hashes for git dependencies, these dependency definitions need to be combined with information from the internet. There are multiple solutions that I see being used:

generating a .nix file describing the dependencies including all needed hashes and including it in the packaging (gemset.nix, yarn.nix)
downloading the dependencies during build time with a fixed-output derivation (jd-gui package, rust crates)

Which one is actually more favorable? The first one results in difficult-to-maintain packages and spams nixpkgs with large files, the second one is not strictly pure, and it is kind of working around nix using the fixed-output derivation, I think I read edolstra discouraging it.
There was some discussion on IRC, but there was no conclusion, so I raise the issue here, because I think there should be a consensus on how to handle this problem in nixpkgs.
Maybe there is an even better third approach that I don't know yet.

Approach 2 (ab)uses fixed-output derivations. Why this might be a bad idea is described here: https://github.com/NixOS/nix/issues/2270

Glossary:

LPM: $language package manager

Here is a brain dump of what I know so far:
0. Fixed output derivations
The fixed-output derivation is the level zero support. It doesn't take much effort to create and maintain. The language package manager (LPM) commands can be used directly like in the developer documentation.
The biggest downside is that the hash is not automatically invalidated when one of the input files are changing. This can create surprising situations when the lockfile is updated but the old program is still running (because it's still reading from the old hash). The hash has to be invalidated manually but changing it to something else, run nix-build and wait for nix to tell you the right hash. Then re-run the build from scratch.
Another downside is that not all the tools have a stable on-disk output. Two developers not sharing a binary cache might get different output hashes. I've seen that happen with the cargo tools for a while for example.
1. Fake registry
A lockfile is generated that downloads all the dependencies using nix fetchers. Then the aggregate is used to start a fake registry process that the tools can talk to.
This solves the outdated hash problem, and since the APIs are usually publicly documented they are also pretty stable. The only implementation of that idea that I know of is https://github.com/nmattia/napalm
The biggest downside is that we need to build the API for all the languages.
2. Pre-download the dependencies
This is similar to (1) but instead of providing an API, the files are placed on disk where the LPM expects to find them. There is often an offline mode that we can re-use.
In my experience the on-disk locations are no necessarily documented or stable between releases. To implement it properly, the nix developer is often forced to look at the LPM source code to find the location and duplicate the logic in nix.
3. Build each dependency in it's own derivation
This is going even further than (2) in the LPM integration. The LPM usually controls traversing the dependency tree and running each individual builds, which we hijack and replace with individual derivations that are being built. So here each dependency is built in isolation in it's little sandbox, and then stringed together and presented to the application at the end. Even more heuristic of the LPM is encoded into Nix. Examples of that can be found for ruby and python.
The main advantage of this approach is that it minimized the rebuild between two releases. The rebuilds are more incremental. And in theory the dependencies can also be shared between two or more programs.
The main downside is that it's a lot of nix code that is running to build a single package. The Hydra evaluator is now running on a 64GB node because evaluating nixpkgs takes a lot of memory.
And while sharing is nice, in practice ruby/node/rust projects very rarely share exactly the same dependency set.
This is a great approach for a company monorepo. Or when maintaining a package set snapshot like Stackage or the python modules.
For single packages in nixpkgs, I now believe that this is going one step too far.

And while sharing is nice, in practice ruby/node/rust projects very rarely share exactly the same dependency set.

Derivations don't need the exact same dependency set for sharing to be useful though, as long as there are some common dependencies.
IME sharing is very common even cross-projects, especially in nodejs where some dependencies are in almost every larger project.
I quickly checked duplication across gemset.nix in nixpkgs and this was the result: https://gist.github.com/adisbladis/730b982cc6b1a8013581529639c40ce0.

:+1: on adding data to what is essentially a belief right now.
I think you have to show that the derivation outputs are the same. For example addressable-2.6.0 is used in 16 projects but has a dependency on public_suffix >= 2.0.2, < 4.0 . There are 5 different versions of public_suffix in the gist so potentially 5 different derivation outputs for the same addressable-2.6.0 gem. Basically leafs are shared each level below rest is exponentially less likely to be.
That being said, the sharing also happens between multiple versions of nixpkgs. Having more granular derivations also allows to minimize rebuild on package updates, and minimize downloads from the user.
To really know we would need a big differential equation that balances build times, evaluation times and download times.
Actually I was missing the last step:
4. Use Nix as a project build system
In this scenario, there is no LPM. Nix has entirely replaced the LPM tooling. Nix is building each an every object of a project in it's own derivation and composing them all together. This is the ultimate incremental rebuild, and the ultimate memory and Nix evaluation hog.
At that point you very much wish that Nix had an Intentional Store to minimize rebuilds.

The issue then becomes that you have to put generated nix files in git, and these can be very large. Do you have any ideas how this could be solved? I had some ideas about using git lfs or some content addressable storage, like ipfs.

@offlinehacker This could potentially be addressed by nix flakes (& splitting up nixpkgs into subsystems).

Also there's one other option that's variation of fake registry option described above:
Recording/Caching http proxy
You redirect all requests of LPM through local http proxy. This proxy records all requests and transforms in a way that can be later used for reply during installation process. The problem is that package manager not only loads tar archives and git repos, but also makes api requests to something like npm. You need to make response transformations that are specific to each package manager, but the whole service could be generalized with plugins.
During installation process you start proxy again with generated configuration from first step as an input.
The benefit is that you now no longer require fake registry for every package manager but you have more generalized solution.

@adisbladis in any case even if you split repo, you still polute other repos with basically files that are large text blobs, but yeah I agree that this would still help. The problem is we can't package some things because generated files are too large, for example take a look here: https://github.com/NixOS/nixpkgs/pull/49082

The issue then becomes that you have to put generated nix files in git, and these can be very large. Do you have any ideas how this could be solved? I had some ideas about using git lfs or some content addressable storage, like ipfs.

The best solution that I know of is to extend the nix capabilities to allow recursive nix calls. Recursive Nix is when nix is being called from inside a derivation. I would look a little bit like this:
stdenv.mkDerivation {
  pname = "xxx";
  version = "1.2.3";
  src = fetchFromGitHub { ... };
  buildPhase = ''
    nix-build -I nixpkgs=${pkgs.path} ./default.nix
     # or ${./inner.nix} if upstream doesn't have a default.nix or we don't want to use it
  '';
  installPhase = ''
    ln -s $(readlink ./result) $out
  '';
}

(obviously we would extract this pattern in a new pkgs.mkRecursive function)
The nice thing here is that import-from-derivation can be allowed in the inner build. It's not going to affect nix-env -qaP. And the lockfile is sourced directly from upstream instead of having to duplicate it in nixpkgs. And if upstream has already packaged the project with nix we can also defer that to them (except the meta and passthru attributes).
So overall it would make hydra builds a bit slower because nixpkgs has to be re-evaluated again on each build. For the users, the nixpkgs evaluation becomes faster because the complicate IFD happens only at build time. If we start using nix files from upstream it might make refactoring of nixpkgs a bit harder. Package dependencies are harder to follow since they are not passed to the outer default.nix.

Recursive nix was also discussed in this RFC: https://github.com/NixOS/rfcs/pull/40

When there's something I can try and maybe an example of how to use it, I would love to start experimenting with it to get rid of all the yarn.nix files, where possible.
However, I do not see how this could solve the issue for example with ruby tooling, where the hashes are not included in the lockfile.

The topic of how to do language package managers came up in #78810 again.
@Mic92 described another issue that was not considered here, which is time and memory required for evaluating nixpkgs (like when doing nix-review).
I am wondering: Would this issue be solved by recursive nix?

By the way: This pattern could avoid the expression size explosion: https://github.com/NixOS/nixpkgs/pull/87258/files#diff-97ddd5942a260ac035c022c0c57de234R20

Originally posted by @Mic92 in https://github.com/NixOS/nixpkgs/pull/78810#issuecomment-625808078
I think this discussion should not be held in the Mastodon PR, because it is not specific to mastodon or even yarn2nix. The bundler tooling and some Go tooling works the same and has the same issues.
I think this is just moving the problem from expression size / evaluation speed to hidden impurities (see https://github.com/NixOS/nix/issues/2270).
This is a general issue and I think it might even be good to create some kind of working group of people who are interested in finding a community concensus and solving this problem for all language package managers in the long-term. I would certainly be interested in it.

This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/future-of-npm-packages-in-nixpkgs/14285/3


[Bug] Attempting to upload a manual log file on TC-DGTHREAD-2.1 results in a "Failed to connect' error and partial log

From chip-certification-tool created by jrhees-cae: CHIP-Specifications/chip-certification-tool#970
Describe the bug
When attempting to upload a manual log file on TC-DGTHREAD-2.1 we receive a "Failed to connect' error and only a portion of the log is included.
Steps to reproduce the behavior
Run TC-DGTHREAD-2.1
At end of test, upload attached log file
Log file can be found here: https://csamembers.slack.com/files/U037HSS0V2P/F05SSE1DTF0/tc-dgthread-2.1.txt
Expected behavior
No response
Log files
No response
PICS file
No response
Screenshots
No response
Environment
No response
Additional Information
No response

Hi @jrhees-cae is this issue 100% reproducible?
In my setup I was able to execute and upload the attached tc-dgthread-2.1.txt file.
This is my execution log (All manual logs are printed inside this file)
UI_Test_Run_2023_09_19_13_17_20.log
UI With manual log printed

We tried running the test 3 or 4 times and got the same error every time. Then we ran a different test (with a different log) and it worked fine. We haven't tried since.

Hi @jrhees-cae the informed log tc-dgthread-2.1.txt have some chars that are not decodable to UTF-8. The lines are related to the output to docker ps command.

I removed the output from docker ps command from tc-dgthread-2.1.txt log file, and performed the test again and the upload was successfully.
New Log File with removed lines: TC-DGTHREAD-2.1 (1)V2.txt


How to edit number of event in a day?

Good morning,
I would like to change the number of items that are displayed in a single day. The single item has been customized as pictured. Is there any way to change this number? I would like to change it after the third event (so 3 events and then the +X events button)

The number of events, displayed within a day, is completely dependent upon the available space at the time of render.


Modify defaults for scan-interval, scale-down-unneeded-time

Towards https://github.com/giantswarm/giantswarm/issues/11157, https://github.com/giantswarm/giantswarm/issues/11133
Changes

Increase scan interval from 10 to 30 seconds
Release unneeded nodes after 5 instead of 10 minutes

Tested, works as expected.


mergeable and parallization

hi hypertwobits team,
Just wonder whether it is mergeable as the normal HLL, e.g., I sketch the first part of the file and then I sketch the second part and merge sketch in the end to estimate cardinality of the big file. This has many applications in bioinformatic and genomics, since there may be billions of elements in a file and if I want to parallelize it using multiple threads to process the file and then merge to count.
Thanks,
Jianshu

Forget it. I saw the merge function.


Some data fields have type of f64 but should have Decimal

Some struct has type of f64 instead of Decimal.
The f64 has rounding issues and is not suitable for this use case.
Possible invalid structs

TradeTransInfo
TradeTransactionStatusResponse
StreamGetBalanceData
StreamGetCandlesData
StreamGetProfitData
StreamGetTickPricesData
StreamGetTradesData
StreamGetTradeStatusData

done


WIP: First simple implementation

[x] tests and linter show no problems (npm t)
[x] tests are added/updated for bug fixes and new features
[x] code is properly formatted (npm run fmt)
[x] description of changes is added in CHANGELOG.md
[x] update .d.ts typings

Landed in https://github.com/metarhia/metacalc/commit/613f96c6b9683a27f753e6029fad5e6e8ba5df32


M3 Setup + Content

I've put in all the info that I can think of, but I am sure I will have missed stuff, so lemme know what I should add in. Challenges are on their way, but I have a few ideas that I need to do some reading on before I can flesh them out properly.

I also changed directory structure a little bit for images for both the m3 section and the getting started section, so images are in a subfolder instead of main chapter folder. Makes it a bit cleaner.

Fixed the merge conflict, except I think i did it stupidly my bad

Fixed the merge conflict, except I think i did it stupidly my bad. I just clicked on the resolve merge thing in github and edited it there instead of just editing my branch so it merged from dev to m3??? idk how this stuff works completely

That's okay. It The Git graph looks fine. Everything is merging into dev anyway and there are going to be merge conflicts anyway from all the branches merging together which I will be fixing. I'll review the changes now.


Split out projects, pages and guides in codebase

Working on #11073, one of the problems we have is that projects and guides are rendered using the pages getServerSideProps (and therefore transformPage) and the Page component, although they contain different fields.
To simplify it in the future, I want them to completely be split in the codebase.

In prod


[SPARK-20024] [SQL] [test-maven] SessionCatalog API setCurrentDatabase need to set the current database of ExternalCatalog [WIP]

What changes were proposed in this pull request?
SessionCatalog API setCurrentDatabase does not set the current database of the underlying ExternalCatalog. Thus, weird errors could come in the test suites. We need to fix it.
So far, have not found the direct impact in the other code paths because we expect all the SessionCatalog APIs should use the current database value we managed, unless some of code paths skip it.
How was this patch tested?
Multiple test case failures are observed in mvn
TODO: add a test case in SessionCatalogSuite.

Test build #74850 has started for PR 17354 at commit c552c78.

retest this please

Test build #74854 has started for PR 17354 at commit c552c78.

Test build #74854 has finished for PR 17354 at commit c552c78.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74854/
Test FAILed.

Merged build finished. Test FAILed.

retest this please

Test build #74859 has started for PR 17354 at commit c552c78.

retest this please

retest this please

retest this please

Test build #74860 has started for PR 17354 at commit c552c78.

Test build #74860 has finished for PR 17354 at commit c552c78.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74860/
Test FAILed.

Test build #74850 has finished for PR 17354 at commit c552c78.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74850/
Test PASSed.

@gatorsmile Hi Sean, with this fix, do we still require the other fix in OrcSourceSuite ? We know that it can't hurt.. but it helps expose problems like this ?

Yes, we can revert it back

@gatorsmile Thanks a lot.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74859/
Test FAILed.

retest this please

Test build #74865 has started for PR 17354 at commit c552c78.

retest this please

Test build #74868 has started for PR 17354 at commit c552c78.

Test build #74868 has finished for PR 17354 at commit c552c78.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74868/
Test FAILed.

Test build #74865 has finished for PR 17354 at commit c552c78.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74865/
Test FAILed.

retest this please

Test build #74892 has started for PR 17354 at commit 31f1099.

Test build #74892 has finished for PR 17354 at commit 31f1099.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74892/
Test FAILed.

Test build #74900 has started for PR 17354 at commit e860fd7.

Test build #74902 has started for PR 17354 at commit f23984d.

Test build #74902 has finished for PR 17354 at commit f23984d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74902/
Test FAILed.

retest this please

Test build #74904 has started for PR 17354 at commit f23984d.

Test build #74900 has finished for PR 17354 at commit e860fd7.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74900/
Test FAILed.

Test build #74904 has finished for PR 17354 at commit f23984d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74904/
Test PASSed.

cc @cloud-fan @yhuai @windpiger

Test build #74915 has started for PR 17354 at commit d6c63b3.

LGTM

Test build #74915 has finished for PR 17354 at commit d6c63b3.

This patch fails PySpark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74915/
Test FAILed.

retest this please

Test build #74928 has started for PR 17354 at commit d6c63b3.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/74928/
Test PASSed.

Thanks! Merging to master.

late LGTM~


Error reported by XML parser: Invalid byte 2 of 3-byte UTF-8 sequence.

Hello
Thank's for your tool
I'm getting the same error with a lot of pdf files
I'm trying the grobid service 0.4.1 on a windows 10 install
A sample PDF file is available here
http://givingsense.eu/sembib/data/srcPdf/A Branch and Bound Method for the Aggregation of Symmetric Relations.pdf
which is accepted by several PDF tools
Here is the begining of the log for a failing pdf
[DEBUG] org.grobid.core.engines.Engine: Ending processing fullTextToTEI on C:\outils\SemanticWeb\grobid-grobid-parent-0.4.1\grobid-home\tmp\origin3628539216410344526.pdf. Time to process: 22525ms
Error on line 46 column 48
SXXP0003: Error reported by XML parser: Invalid byte 2 of 3-byte UTF-8 sequence.
[ERROR] org.grobid.service.process.GrobidRestProcessFiles: An unexpected exception occurs.
net.sf.saxon.trans.XPathException: org.xml.sax.SAXParseException; lineNumber: 46; columnNumber: 48; Invalid byte 2 of 3-byte UTF-8 sequence.
at net.sf.saxon.event.Sender.sendSAXSource(Sender.java:418)
at net.sf.saxon.event.Sender.send(Sender.java:193)
at net.sf.saxon.event.Sender.send(Sender.java:50)
at net.sf.saxon.Configuration.buildDocument(Configuration.java:2973)
at net.sf.saxon.query.StaticQueryContext.buildDocument(StaticQueryContext.java:211)
at org.grobid.core.utilities.XQueryProcessor.(XQueryProcessor.java:48)
at org.grobid.core.utilities.BibDataSetContextExtractor.getCitationReferences(BibDataSetContextExtractor.java:56)
at org.grobid.core.visualization.CitationsVisualizer.getJsonAnnotations(CitationsVisualizer.java:248)
at org.grobid.service.process.GrobidRestProcessFiles.processPDFReferenceAnnotation(GrobidRestProcessFiles.java:727)
at org.grobid.service.GrobidRestService.processPDFReferenceAnnotation(GrobidRestService.java:649)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
...

Hello !
The PDF file is working fine with current master version (0.4.2-SNAPSHOT). The pdf parser has been improved in between (it uses now the following fork: https://github.com/kermitt2/pdf2xml and is more robust).
You can test it with the latest version via the online demo: http://grobid.science-miner.com
The current master with this new pdf parser is not yet available on Windows, some binaries need to be recompiled for Win64.

You can however make the master version work on Windows 10 as explained by @UntoterOstgote on issue #185.

@moissinac the issue you had with the UTF-8 XML parsing should be now fixed, give it a try and let us know

Tested with success with the online version and the service version for the latest release
Many thank's


Dashboard: Safeguard against NaN page numbers to fetch stories

Summary
This is a weird one that i cannot reproduce unless i force the environment and hardcode the wrong value for the page - I've completely reset my local database to clear everything out and follow steps to reproduce.
The gist of the bug is that the API for stories was breaking because the page number passed in for the query was NaN.
/web-story?context=edit&order=desc&orderby=modified&page=NaN&per_page=24&status=draft&_locale=user
(where page=NaN)
The default page value is 1 and if it is undefined when it gets passed to fetch the stories it is set as 1. So somewhere in this reported bug the page value is becoming something other than an integer on that initial load. To prevent against this I am checking that the page value is typeof === number as we go to make the API call and if not making sure 1 is the page number passed to the query.
Relevant Technical Choices

Check that page value is indeed a number, if it is not pass 1 as page query

To-do

If page is NaN when it gets to the API it could be set to page 1 in the dashboard controller instead.

User-facing changes

Should be an invisible change unless you were seeing this bug!

Testing Instructions

See that the first page of stories loads in your dashboard when you navigate to it.

Addresses #2950

@BrittanyIRL Since #2950 turned out to be a non-issue, maybe best to close this PR without merging?

@BrittanyIRL Since #2950 turned out to be a non-issue, maybe best to close this PR without merging?

Agreed!


Bash variables declaration should not use "$"

Hello,
there is a mistake in the az-cli examples:
$subscriptionId=$(az account show --output=json | jq -r .id)
should look
subscriptionId=$(az account show --output=json | jq -r .id)
Thank you
Regards
Roberto

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: f8cff5bf-cbe3-0a0b-f972-f0dca4c39258
Version Independent ID: 801b467a-898d-d4d2-82d0-636aa7678752
Content: Enable and disable the Azure Serial Console - Virtual Machines
Content Source: support/azure/virtual-machines/serial-console-enable-disable.md
Service: virtual-machines
GitHub Login: @genlin
Microsoft Alias: genli

@rtorresa, Thanks for the feedback, I will fix the issue.

@rtorresa If you have any issue with the Azure Serial Console, you can also contact Azure support for assistance.

@rtorresa If you have any issue with seeraio cont


is there any flag/callback function i can use to determine the events are loaded/refreshed?

since i saw eventdidmount function won't be called when event is refresh, i think i can't know when did the calendar loaded. (i want to do some programmatic click event on calendar)
what i want to do is just, when calendar refetch the events and finish updating doms, i'd like to pick some events out and add some class into dom.  what can i do?

Won't eventContent work for you in this case?

ya eventContent maybe work. acutally what i want to do is:
i have some events, and i want to pick the most recent one before today. So if i pre-mark the most recent event , maybe eentContent can recognize the date i want to update. but acutally what i want to do is if i find the date, i would click the event on it to trigger the eventClick. ... oh maybe i could do click in eventContent ... although it sounds strange 😂

let me try and see if i could close this issue

Any success?

@vkurko
Really don't understand why there is no callback fired when all events are rendered. This functionnality was awesome in the v3 and many people need this

@vkurko Really don't understand why there is no callback fired when all events are rendered. This functionnality was awesome in the v3 and many people need this

Ok, I'll implement it.

Hi all. Sorry for delay.
Version 2.6.0 introduced an experimental eventAllUpdated option, which may be useful for your use cases. It turned out to be not so easy to catch the end of updates of all events within the reactive framework. But I think that even with the current behavior, it is already possible for you to achieve the desired results. Please check.

I hope I can close this issue.


Error: Not able to find or install a local wasm-bindgen.

Hi, I found an error when running yarn start.
rustup --version: 1.23.1
rustc: 1.48.0
[INFO]: Compiling to Wasm...
Compiling proc-macro2 v1.0.18
Compiling unicode-xid v0.2.1
Compiling syn v1.0.34
Compiling log v0.4.11
Compiling wasm-bindgen-shared v0.2.65
Compiling cfg-if v0.1.10
Compiling bumpalo v   Compiling bumpalo v3.4.0
Compiling lazy_static v1.4.0
Compiling wasm-bindgen v0.2.65
Compiling serde v1.0.114
Compiling ryu v1.0.5
Compiling serde_derive v1.0.114
Compiling serde_json v1.0.56
Compiling itoa v0.4.6
Compiling quote v1.0.7
Compiling wasm-bindgen-backend v0.2.65
Compiling wasm-bindgen-macro-support v0.2.65
Compiling wasm-bindgen-macro v0.2.65
Compiling js-sys v0.3.42
Compiling web-sys v0.3.42
Compiling wiesn v0.1.0 (/home/ngh/Desktop/wiesn)
warning: unreachable expression
--> src/lib.rs:126:9
|
126 |            return match MALTS.iter().find(|malt| malt.id == malt_id) {
|  ____^-
| | |
| ||
127 | ||             None => return Recipe {
128 | ||                 malts_in_recipe,
129 | ||                 volume,
...   ||
145 | ||             }
146 | ||         }
| ||         ^
| |||
| |__________unreachable expression
|            any code following this match expression is unreachable, as all arms diverge
|
= note: #[warn(unreachable_code)] on by default
warning: unreachable expression
--> src/lib.rs:222:9
|
222 |            return match YEASTS.iter().find(|yeast| yeast.id == yeast_id) {
|  ____^-
| | |
| ||
223 | ||             None => {
224 | ||                 let yeast = self.yeast;
225 | ||                 return Recipe {
...   ||
240 | ||             }
241 | ||         }
| ||         ^
| |||
| |__________unreachable expression
|            any code following this match expression is unreachable, as all arms diverge
warning: 2 warnings emitted
Finished dev [unoptimized + debuginfo] target(s) in 14.60s

[INFO]: Installing wasm-bindgen...
Error: Not able to find or install a local wasm-bindgen.

I was able to run it by checking my wasm-bindgen 0.2.69 and add it to the [dependencies] wasm-bindgen = "0.2.69" in Cargo.toml


improve keysets ui

Changes in Keyset management:

Keyset can be named
can list keysets by site id
Changes in Encryption Key Management:
can list keyset keys, list keys, list keyset keys by keyset id, list keys by site id

Could you help improve the code on com.uid2.admin.util.PrivateSiteUtil#getKeysetKeys and don't crash if
globalKeysets.get(keysetKey.getKeysetId()) is not found?

For the issue i mentioned above, could you help improve the code on com.uid2.admin.util.PrivateSiteUtil#getKeysetKeys and don't crash if
globalKeysets.get(keysetKey.getKeysetId()) is not found?
And use the logger to log an error if it is indeed not found.
Please look at existing tests and see if you can add one for this as well

For the issue i mentioned above, could you help improve the code on com.uid2.admin.util.PrivateSiteUtil#getKeysetKeys and don't crash if
globalKeysets.get(keysetKey.getKeysetId()) is not found?
And use the logger to log an error if it is indeed not found.
Please look at existing tests and see if you can add one for this as well

I have put a check for this in PrivateSiteUtil and have added a KeysetKeys test section with a couple tests including for this scenario.


Implement __len__() in wrapped Riak page objects

The underlying Riak client library objects implement __len__() so this should be possible. This should be a small optimisation for counting the length of items in a page (which doesn't happen that often).
Class of interest is vumi.persist.riak_base.VumiIndexPageBase

:+1:


Ensure build/build workflow always runs after docker/build is done

Reduces margin of human error.

Closing because Github workflow supports such a defer feature (yet)


Added tax document functions

Also, general comment. Could you also add an 1120S sample file? I don't think there are any available in the documentation.

Also, general comment. Could you also add an 1120S sample file? I don't think there are any available in the documentation.

Did you intend for the colabs to be added to this Pull request as well? I'm not sure they belong here.


[Bug] Unable to access the app at port 8000 when using the development server of app

Is there an existing issue for this?

[X] I have searched the existing issues

What projects are you seeing the problem on?
OpenChallenges
Current behavior
The  page (http://localhost:8000/) should be redirected to the app port (4200) using development mode (nx serve openchallenges-app)  based on the configuration in apex. However, the page shows "502 Gateway Error" on the http://localhost:8000/, whereas the zipkin (http://localhost:8000/zipkin) and api-gateway (e.g. http://localhost:8000/api/v1/challenges) still work.
The app can be accessed using the container of the app (nx serve-detach openchallenges-apex).
Expected behavior
Developers should be able to access the app using development mode at port 8000.
Anything else?
No response
Commit ID
No response
Are you developing inside the dev container?

[X] I am using the dev container

Code of Conduct

[X] I agree to follow this project's Code of Conduct

The issue is related to how the Apex, which is an Nginx reverse proxy, handles the host names of the component. Here are the host names and port that we define in the .env of the apex:
API_GATEWAY_HOST=openchallenges-api-gateway
API_GATEWAY_PORT=8082
APP_HOST=openchallenges-app
APP_PORT=4200
THUMBOR_HOST=openchallenges-thumbor
THUMBOR_PORT=8889
ZIPKIN_HOST=openchallenges-zipkin
ZIPKIN_PORT=9411

When starting the Apex, the container openchallenges-app is started first because we defined that the apex depends on the app container. When the apex container then starts, Docker translates the host name openchallenges-app to the Docker IP address of the app container. if the app container no longer exists and we start the app in development mode, the apex will still forward the requests to a Docker IP that is no longer valid, hence the Bad Gateway issue.
Apex fails to start when its Nginx config still references openchallenges-app WHILE the app container is not running. In the example below, we have removed the dependency of the Apex container on the app container in the Docker Compose file of the apex, so the app container is not running when starting apex.
$ nx serve-detach openchallenges-apex
$ docker logs -f openchallenges-apex
...
20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/events.conf.template to /etc/nginx/conf.d/events.conf
20-envsubst-on-templates.sh: Running envsubst on /etc/nginx/templates/http.conf.template to /etc/nginx/conf.d/http.conf
/docker-entrypoint.sh: Launching /docker-entrypoint.d/30-tune-worker-processes.sh
/docker-entrypoint.sh: Configuration complete; ready for start up
2023/07/12 23:34:58 [emerg] 1#1: host not found in upstream "openchallenges-app:4200" in /etc/nginx/conf.d/http.conf:11
nginx: [emerg] host not found in upstream "openchallenges-app:4200" in /etc/nginx/conf.d/http.conf:11

The logs show that the upstream "openchallenges-app:4200" could not be found. The same error occurs if the app server runs in development mode when the apex is started.
This issue does not affect backend services
The follow works:

Start the stack with nx serve-detach openchallenges-apex
Remove the container of the challenge service with docker rm -f openchallenges-challenge-service
Start the challenge service in development mode with nx serve openchallenges-challenge-service
Try to access localhost:8000/api/v1/challenges still works

The reason is because the Apex sends the requests to the backend via the API gateway. The API gateway then sends the request to the services. Because we don't touch to the API gateway, the apex has no issue. Moreover, it does not matter to the API gateway whether the challenge service is running with its development server or with its container.
Solution
I need to do some exploration work to enable the apex container to access the development server of the app. It shouldn't be a blocker for development for now, as we should still be able to access the development server of the app on localhost:4200.

@rrchai I don't think that there is a simple solution to this "issue". I've came across a few solutions but they may necessarily increase the complexity of the dev environment.
When running the development server of the web app, the idea is to access it on the port the dev server exposes instead of the apex port (8000). This approach should work perfectly fine. If you bump into an issue, feel free to reopen this ticket or open a new one.