Capturing vague units of time

I recently annotated a cyology-related blog and noticed that grobid doesn't allow for vague or inexplicit units of time to be captured. Examples of these include: late July, early August, end of the month, this week, through April, recent decades etc.
I also noticed that it ignores mentions of seasons like spring, fall, summer, summertime, winter, wintertime. Cryology has it's own unique terms to denote seasons like melt season or ice growth period.
It would be very useful if grobid 1)could capture these vague time expressions, 2) if it could be linked to the document/blog/articles publishing date, and 3) if grobid allowed prototypical seasons (if not also those unique to cryology season terms) to be captured as a kind of time expression.

Dear @JenPreciado, apologise for this very very late answer, it is indeed long time since you opened this issue.
In general, the amount of training data available for grobid-quantities, as of today, is still quite limited, and cover mostly papers from astronomy, health and physics. Said that we are quite happy for the current performances, however we are aware there is needs for more data.
We haven't really focused on recognising vague expressions because there was the problem, afterwards on how to normalise them (e.g. expressions like in recent time) . We have decided to leave the vague part of the expression out of the annotation e.g. later <date>July</date> and focus on resolving the expression itelf.
BTW The guidelines annotations (https://grobid-quantities.readthedocs.io/en/latest/guidelines.html) have been improved in the last year with more examples and special cases.  In brief we annotate time/date expressions as <date when="2001-08">2001 August</date> (https://grobid-quantities.readthedocs.io/en/latest/guidelines.html#additional-items).
If you are still working on it and you want to share what you have done, we can see whether there are some complementary needs.


epic: Finalize how Model Folder and model.yaml works

Goal

We should have a model folder that is able to handle different models

Built-in models (e.g. janhq/llama3:7b-tensorrt-llm)
Huggingface GGUF repos with multiple quants (e.g. bartowski/llama3-gguf)
Huggingface specific GGUF (may have multiple from same directory)
In future: Nvidia NGC or TensorRT Cloud

Do we use sub-folders?
How does model.yaml work?
Model detection should not depend on model folder

Decisions

https://github.com/janhq/cortex.cpp/discussions/1113
https://github.com/janhq/cortex.cpp/discussions/1123
#1178

Edge Cases

What if we download multiple GGUFs from the same Huggingface repo?
How does cortex model update <model> work?
Legacy model folder structure: https://github.com/janhq/jan/issues/3541#issuecomment-2328413473

Tasklist

[ ] Lessons learned from Jan's model folder

Legacy model folder structure: https://github.com/janhq/jan/issues/3541#issuecomment-2328413473

Model detection should not depend on model folder?
Model detection depends on the model folder, which would introduce performance issues since:

Every time the app loads, it needs to scan through the model folder hierarchy.
Costly filesystem watching (Notify the app of changes)

This means we likely depend on the manifest file, where it is the source of trust, as it links to all available models with different folder structures.

This introduces a periodically scanning folder watchdog. A delay problem may occur.
Everything just works with references or symlinks.

Structures do not work well in the past or I have seen around:
1. Shallow structure
All of the YAML files are placed in the root of the directory.
Pros: Fast lookup - just filter out YAML files from the root folder to list models.
Cons: Easy to duplicate, cannot work with different model families. Same name for different branches/authors/engines.
Slower over time since n models = 2n items. 2 rm operations to remove a model.
E.g. llama3 of cortexhub | gguf | Q4 | Q8 | onnx | thebloke

/models
    /[model1]
       /[model1].bin
       /[model1].gguf
    /[model1].yaml | json

2. One level of deep structure
All files are placed in a model folder.
Pros: Easy to manage model by model, 1 rm operation can remove the entire model folder.
Cons: Slow list iteration, the app has to loop through every single folder and check if the model file exists. So many FS operations.

/models

    /[model1]
       /[model].bin
       /[model].gguf
       /[model].yaml | json

Getting our Filesystem hierarchy less wrong with these 3 following Principles:
Principle 1: The Single-Question Principle
At each level of the hierarchy, strive to make all folder names answer the same question.
Principle 2: The Domain Principle
Organize files in different domains differently.
Principle 3: The Depth Principle
Prefer deep hierarchies over shallow ones.
The structures would be similar to this:
/models
  ├── manifest.yaml
  ├── /metadatas
  │   ├── /[model_id].yaml
  │   ├── llama3.1-7B_Q4_KM.yaml
  │   └── mistral-7B_Q4_KM.yaml
  └── /sources
      ├── /huggingface
      │   ├── /cortexso
      │   │   ├── /llama3-1
      │   │   │   ├── /gguf
      │   │   │   │   ├── /main
      │   │   │   │   │   └── llama3.1_Q4_KM.gguf
      │   │   │   │   └── /7b
      │   │   │   │       ├── llama3.1_Q4_KM.gguf
      │   │   │   │       └── llama3.1_Q8_KM.gguf
      │   │   │   ├── /onnx
      │   │   │   │   └── /7b
      │   │   │   │       ├── llama3.1.onnx
      │   │   │   │       ├── tokenizer.json
      │   │   │   │       └── gen_config.json
      │   │   │   └── /tensorrt-llm
      │   │   │       └── /7b
      │   │   │           ├── rank0.engine
      │   │   │           ├── tokenizer.model
      │   │   │           └── config.json
      │   │   └── /phi-3
      │   │       └── /onnx
      │   └── /bartowski
      │       └── /Llama-3.1-Storm-8B-GGUF
      │           └── /gguf
      │               └── /main
      │                   └── Llama-3.1-Storm-8B-Q4_K_L.gguf
      └── /nvidia-ngc
          └── /llama3-1

At each individual level of the sources hierarchy, all options are different responses to the same question: What hub is the source? What repo is in the hub? What model types are supported in the repo? What branch is the model pulled from? What model and what quantization are pulled?
model.yaml files are flattened for quick search in the metadatas so users can easily find the one they want to edit, which boosts the performance of model listing. The filename is a normalized form of model_id.
From the sources folder hierarchy, we can determine the author andengine, so we can get rid of model.yaml's redundant fields.
Files are organized differently in different domains (metadatas / sources).
Everything is a symlink, from [model].yaml, we can retrieve the source hierarchies.
Manifest is for caching (optional), which can improve UX and boost performance. This allows us to avoid including all computed fields (such as decorations, sorting order - drag and drop later, or sorting results) in model.yaml (e.g., size, quantization). These fields are not essential when constructing or modifying model.yaml, but they do increase the risk of errors. Since they can be retrieved from the source files, we only need to cache them when populating the model.

cc @dan-homebrew @0xSage

Model Folder
I like the approach recommended by /etc/apt/sources.list, and agree with the following principles:

Manifest file > file system watching
Single question, domain, depth principles

I would like to brainstorm a few simplification ideas:
Using "pull name" as folder name
I wonder if this is more user recognizable, vs. multiple nested folders.
/models
    models.list (index)
    /llama3.1
         llama3.1.gguf
    /llama3.1:tensorrt-llm
         ...
    /huggingface.co/bartowski/llama3.1-gguf-7b
         llama3.1-7b-gguf

models.list
A lot of how effective this will be, depends on models.list format. Need to articulate how that will work.
Optional model.yaml or <model_name>.yaml in model folder

It is still highly beneficial for the model.yaml to be in same folder as source files, for packaging and proximity purposes.
What we should clarify is that model.yaml is optional (especially for GGUF files)
We should also be agnostic to whether it's called model.yaml or <model_id>.yaml (however - need to protect against edge case where there are multiple .yaml files in the model folder)

I like suggestion 1. Most flexible, i.e. model binaries can be anywhere.

model.yaml Structure
I agree with @louis-jan suggestions above.
However, I'm a bit mixed on what sources should refer to:

Individual files? (very tedious to upkeep the model.yaml
Repo? (i.e. collection of files/tags)

Given that our main integration is with Huggingface, and our own Built-in repos use Git, I think Repos would be a better abstraction.

I like suggestion 1. Most flexible, i.e. model binaries can be anywhere.

We did try this approach before with Cortex, but the colon : is not allowed. It turned out that the model folder is not really the pull name as designed.

I like suggestion 1. Most flexible, i.e. model binaries can be anywhere.

We did try this approach before with Cortex, but the colon : is not allowed. It turned out that the model folder is not really the pull name as designed.

Built-in Model Library
I see. In that case, can we consider just have a 2-deep file structure?

1st level: "pull name"
2nd level: tag

llama3.1
    7b

Huggingface Repos

For Huggingface repos, folder can just be a string (can it accomodate / in folder name?)

I like suggestion 1. Most flexible, i.e. model binaries can be anywhere.

We did try this approach before with Cortex, but the colon : is not allowed. It turned out that the model folder is not really the pull name as designed.

Built-in Model Library
I see. In that case, can we consider just have a 2-deep file structure?

1st level: "pull name"
2nd level: tag

/llama3.1
    /7b

Huggingface Repos

For Huggingface repos, folder can just be a string (can it accomodate / in folder name?)

Love it!

I'll summary the implementation for model folder and model.yaml, break it into tasks
/models
├── model.list
└── /llama3-1
|   ├── /main
|   |   ├── model.yaml
|   |   └── llama3.1_Q4_KM.gguf
|   └── /7b-gguf
|   |   ├── llama3.1_Q4_KM.yaml
|   |   ├── llama3.1_Q4_KM.gguf
|   |   ├── llama3.1_Q8_KM.yaml
|   |   └── llama3.1_Q8_KM.gguf
|   ├── /onnx
|   │   ├── llama3.1.yaml
|   │   ├── llama3.1.onnx
|   │   ├── tokenizer.json
|   │   └── gen_config.json
|   └── /tensorrt-llm
|   |   ├── llama3.1.yaml
|   |   ├── rank0.engine
|   │   ├── tokenizer.model
|   └───└── config.json
└── /bartowski-Mixtral-8x22B-v0.1 #huggingface repos "/" will be replace by "-" or orther special character
|   └── /main
|       ├── Mixtral-8x22B-v0.1-IQ3_M.yaml
|       ├── Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf
|       └── Mixtral-8x22B-v0.1-IQ3_M-00002-of-00005.gguf
└── /nvidia-ngc
└── /llama3-1-windows-RTX3090
└──model.engine
model.list content:
model-id author_repo-id branch-name path-to-model.yaml model-alias

How model-id is constructed author_repo-id_branch-name_gguf-file-name (why need gguf-file-name, because under a branch can have multiple gguf files/models with different quant)
model-alias is shorter name for model-id and also unique, user can set alias with command cortex-cpp model alias model_id model_alias, then model_alias should work exactly like model_id

Model.yaml changed
# BEGIN GENERAL GGUF METADATA
model: gemma-2-9b-it-Q8_0 # Model ID which is used for request construct - should be unique between models (author / quantization)
name: Llama 3.1      # metadata.general.name
version: 1           # metadata.version
sources:             # can be universal protocol (models://) OR absolute local file path (file://) OR https remote URL (https://)
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00002-of-00005.gguf
# END GENERAL GGUF METADATA

# BEGIN INFERENCE PARAMETERS
# BEGIN REQUIRED
stop:                # tokenizer.ggml.eos_token_id
  - <|end_of_text|>
  - <|eot_id|>
  - <|eom_id|>
# END REQUIRED
# BEGIN OPTIONAL
stream: true         # Default true?
top_p: 0.9           # Ranges: 0 to 1
temperature: 0.6     # Ranges: 0 to 1
frequency_penalty: 0 # Ranges: 0 to 1
presence_penalty: 0  # Ranges: 0 to 1
max_tokens: 8192     # Should be default to context length
# END OPTIONAL
# END INFERENCE PARAMETERS

# BEGIN MODEL LOAD PARAMETERS
# BEGIN REQUIRED
prompt_template: |+  # tokenizer.chat_template
  <|begin_of_text|><|start_header_id|>system<|end_header_id|>

  {system_message}<|eot_id|><|start_header_id|>user<|end_header_id|>

  {prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
# END REQUIRED
# BEGIN OPTIONAL
ctx_len: 0          # llama.context_length | 0 or undefined = loaded from model
ngl: 33             # Undefined = loaded from model
# END OPTIONAL
# END MODEL LOAD PARAMETERS

Tasks:

Pull model (check model.list -> create folder -> download model -> create .yaml)
Model get (check model.list -> go to target folder -> read yaml -> return result)
Model list (check model.list -> go to target folder -> read yaml -> return result)
Start model/Run ()
model update
model delete
model alias (new command to set alias for a model id)
Update gguf parser vs yaml parser for new model.yaml template and add more inference params to support llamacpp

Questions / edge cases:

How do we handle branch aliases? i.e. I've seen in some repos 7b also being 7b-gguf, or 7b-q4. Maybe not a concern at this scope? Maybe we just assume that branches are unique for now.
I think model-alias will confuse users. Internally, do we intend to use model-alias interchangeably with model-id? In with case, does it make more sense for model-id to be a uuid, which users should never change, that way we are guaranteed uniqueness & persistence.

@dan-homebrew @nguyenhoangthuan99 I just read back comments. This is the one we want to NOT do that is to hack paths together.
E.g. bartowski_Mixtral-8x22B-v0.1
Anyone can break the app by creating two repositories as below:

bartowski_/Mixtral-8x22B-v0.1
bartowski/_Mixtral-8x22B-v0.1

We introduce models.list is to NOT worry about nested levels. E.g. import from other applications.

For Huggingface repos, folder can just be a string (can it accomodate / in folder name?)

Questions / edge cases:

How do we handle branch aliases? i.e. I've seen in some repos 7b also being 7b-gguf, or 7b-q4. Maybe not a concern at this scope? Maybe we just assume that branches are unique for now.
I think model-alias will confuse users. Internally, do we intend to use model-alias interchangeably with model-id? In with case, does it make more sense for model-id to be a uuid, which users should never change, that way we are guaranteed uniqueness & persistence.

This feature should be added, I think we can do it after model folder and model.yaml is stable.
currently, model_id is used for running model, start a model cortex-cpp run <model_id>. But when we want to support run model from many sources with many different cases, we have to make the model_id not only human-readable for user but also unique, but it turn out in some cases model_id is too long bartowski_Mixtral-8x22B-v0.1_Mixtral-8x22B-v0.1-IQ3_M so we decided to make alias command that allow user to make it shorter. But as Louis commented above bartowski_Mixtral-8x22B-v0.1_Mixtral-8x22B-v0.1-IQ3_M still cannot unique.

I'm comming up a solution like docker, when start a container, the container ID is uuid just like model_id as Nicole recommended, and the name will be random. User can set name for that container but the name should be unique, can we implement this?

@louis-jan Yeah, I think you are right.
I think a central problem is that our ability to pull from different sources, leads to different model folder formats:

Cortex "Model Repo" format (tag based)
Huggingface GGUF models (multiple quantizations in a single repo)

Or is there a more generalizable way to deal with this?
/models
    model.list

    # Built-in Library Model Folder format
    /cortex (this is our built-in library)
        /llama3.1 (model)
            /q4-tensorrt-llm **(tag)**
                model.yaml
            /q8-gguf **(tag)**
                model.yaml

    # Huggingface GGUF Model Folder format
    /huggingface
        /bartowski
            /Mixtral-8x22b-v0.1-gguf
                /q4
                    model.yaml
                    Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf
                /q8

    # Future Model Source
    # Has its own model folder format

@louis-jan @nguyenhoangthuan99 Additionally, for model.yaml, how do we intend to generate the model ID?

Is there a way we use the tag name as the model ID?
e.g. for Chat Completions, it is routed to model: llama3.1:7b

# BEGIN GENERAL GGUF METADATA
model: gemma-2-9b-it-Q8_0 # Model ID which is used for request construct - should be unique between models (author / quantization)
name: Llama 3.1      # metadata.general.name
version: 1           # metadata.version
sources:             # can be universal protocol (models://) OR absolute local file path (file://) OR https remote URL (https://)
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00002-of-00005.gguf
# END GENERAL GGUF METADATA

# BEGIN INFERENCE PARAMETERS
# BEGIN REQUIRED
stop:                # tokenizer.ggml.eos_token_id
  - <|end_of_text|>
  - <|eot_id|>
  - <|eom_id|>
# END REQUIRED
# BEGIN OPTIONAL
stream: true         # Default true?
top_p: 0.9           # Ranges: 0 to 1
temperature: 0.6     # Ranges: 0 to 1
frequency_penalty: 0 # Ranges: 0 to 1
presence_penalty: 0  # Ranges: 0 to 1
max_tokens: 8192     # Should be default to context length
# END OPTIONAL
# END INFERENCE PARAMETERS

# BEGIN MODEL LOAD PARAMETERS
# BEGIN REQUIRED
prompt_template: |+  # tokenizer.chat_template
  <|begin_of_text|><|start_header_id|>system<|end_header_id|>

  {system_message}<|eot_id|><|start_header_id|>user<|end_header_id|>

  {prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
# END REQUIRED
# BEGIN OPTIONAL
ctx_len: 0          # llama.context_length | 0 or undefined = loaded from model
ngl: 33             # Undefined = loaded from model
# END OPTIONAL
# END MODEL LOAD PARAMETERS

I agree with the model folder structure, it's dynamic for future model source and we don't have to worry about duplication problem.
For model ID I think we can use tag name with models from cortexso because the model.list has enough information to map model_id with model.yaml file location. e.g.  llama3.1:7b cortex.so/llama3.1/7b/model.yaml
For model ID with model from other source, I think we still need to include enough information for source/author/repo-id/branch/model-file to avoid duplication.

Summary changes for model folder and model.yaml:
Model folder structure
/models
    model.list

    # Huggingface GGUF Model Folder format
    # This assumes we have some sort of quantization selection wizard when downloading?
    /huggingface.co
        /bartowski
            /Mixtral-8x22b-v0.1-gguf (includes all quants)
                Mixtral-8x22B-v0.1-IQ3_M.yaml
                Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf...
                Mixtral-8x22B-v0.1-Q8_M.yaml
                Mixtral-8x22B-v0.1-Q8_M-00001-of-00005.gguf...

    # Built-in Library Model Folder format
    /cortex.so (this is our Built-in Model Library, based on Git, that will be mirrored across a few sites)
        /llama3.1 (model)
            /q4-tensorrt-llm (tag)
                ...engine_files
                model.yaml
            /q8-gguf **(tag)**
                model.yaml

    # Future Model Source
    # Has its own model folder format

model.list
two entries in model.list: model-id /path/to/model.yaml
With model from cortexso: model-id will be structured as repo:tag e.g. llama3.1:7b
With model from other sources: model-id will be structured as source/author/repo-id/branch/model-file
model.yaml
# BEGIN GENERAL GGUF METADATA
model: gemma-2-9b-it-Q8_0 # Model ID which is used for request construct - should be unique between models (author / quantization)
name: Llama 3.1      # metadata.general.name
version: 1           # metadata.version
sources:             # can be universal protocol (models://) OR absolute local file path (file://) OR https remote URL (https://)
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00002-of-00005.gguf
# END GENERAL GGUF METADATA

# BEGIN INFERENCE PARAMETERS
# BEGIN REQUIRED
stop:                # tokenizer.ggml.eos_token_id
  - <|end_of_text|>
  - <|eot_id|>
  - <|eom_id|>
# END REQUIRED
# BEGIN OPTIONAL
stream: true         # Default true?
top_p: 0.9           # Ranges: 0 to 1
temperature: 0.6     # Ranges: 0 to 1
frequency_penalty: 0 # Ranges: 0 to 1
presence_penalty: 0  # Ranges: 0 to 1
max_tokens: 8192     # Should be default to context length
# END OPTIONAL
# END INFERENCE PARAMETERS

# BEGIN MODEL LOAD PARAMETERS
# BEGIN REQUIRED
prompt_template: |+  # tokenizer.chat_template
  <|begin_of_text|><|start_header_id|>system<|end_header_id|>

  {system_message}<|eot_id|><|start_header_id|>user<|end_header_id|>

  {prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
# END REQUIRED
# BEGIN OPTIONAL
ctx_len: 0          # llama.context_length | 0 or undefined = loaded from model
ngl: 33             # Undefined = loaded from model
# END OPTIONAL
# END MODEL LOAD PARAMETERS

Break down to tasks:

[ ] Pull model (check model.list -> create folder -> download model -> create .yaml)
[ ] Model get (check model.list -> go to target folder -> read yaml -> return result)
[ ] Model list (check model.list -> go to target folder -> read yaml -> return result)
[ ] Start model/Run ()
[ ] model update
[ ] model delete
[ ] model alias (new command to set alias for a model id)
[ ] Update gguf parser vs yaml parser for new model.yaml template and add more inference params to support llamacpp

I'll create subtasks corresponding to above task. cc @dan-homebrew @0xSage  @vansangpfiev @namchuai @louis-jan

@nguyenhoangthuan99 This more or less looks correct.
However, can I verify something re model.list. In your earlier post, you referenced that model.list also has a model-alias.

I still think we should have a model alias, and disagree with @0xSage's post
Asking a user to do a model: huggingface.co/bartowski/llama3.1-7b-gguf in a chat completions request is bad UX
I think we should automatically infer the model alias to be llama3.1-7b-gguf
We should have some form of simple checks to ensure no collisions occur, and add a suffix if there is a collision

model-id author_repo-id branch-name path-to-model.yaml model-alias

Summary changes for model folder and model.yaml: Model folder structure
/models
    model.list

    # Huggingface GGUF Model Folder format
    # This assumes we have some sort of quantization selection wizard when downloading?
    /huggingface.co
        /bartowski
            /Mixtral-8x22b-v0.1-gguf (includes all quants)
                Mixtral-8x22B-v0.1-IQ3_M.yaml
                Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf...
                Mixtral-8x22B-v0.1-Q8_M.yaml
                Mixtral-8x22B-v0.1-Q8_M-00001-of-00005.gguf...

    # Built-in Library Model Folder format
    /cortex.so (this is our Built-in Model Library, based on Git, that will be mirrored across a few sites)
        /llama3.1 (model)
            /q4-tensorrt-llm (tag)
                ...engine_files
                model.yaml
            /q8-gguf **(tag)**
                model.yaml

    # Future Model Source
    # Has its own model folder format

model.list two entries in model.list: model-id /path/to/model.yaml With model from cortexso: model-id will be structured as repo:tag e.g., llama3.1:7b With model from other sources: model-id will be structured as source/author/repo-id/branch/model-file
model.yaml
# BEGIN GENERAL GGUF METADATA
model: gemma-2-9b-it-Q8_0 # Model ID which is used for request construct - should be unique between models (author / quantization)
name: Llama 3.1      # metadata.general.name
version: 1           # metadata.version
sources:             # can be universal protocol (models://) OR absolute local file path (file://) OR https remote URL (https://)
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00002-of-00005.gguf
# END GENERAL GGUF METADATA

# BEGIN INFERENCE PARAMETERS
# BEGIN REQUIRED
stop:                # tokenizer.ggml.eos_token_id
  - <|end_of_text|>
  - <|eot_id|>
  - <|eom_id|>
# END REQUIRED
# BEGIN OPTIONAL
stream: true         # Default true?
top_p: 0.9           # Ranges: 0 to 1
temperature: 0.6     # Ranges: 0 to 1
frequency_penalty: 0 # Ranges: 0 to 1
presence_penalty: 0  # Ranges: 0 to 1
max_tokens: 8192     # Should be default to context length
# END OPTIONAL
# END INFERENCE PARAMETERS

# BEGIN MODEL LOAD PARAMETERS
# BEGIN REQUIRED
prompt_template: |+  # tokenizer.chat_template
  <|begin_of_text|><|start_header_id|>system<|end_header_id|>

  {system_message}<|eot_id|><|start_header_id|>user<|end_header_id|>

  {prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
# END REQUIRED
# BEGIN OPTIONAL
ctx_len: 0          # llama.context_length | 0 or undefined = loaded from model
ngl: 33             # Undefined = loaded from model
# END OPTIONAL
# END MODEL LOAD PARAMETERS

Break down to tasks:

[ ]  Pull model (check model.list -> create folder -> download model -> create .yaml)
[ ]  Model get (check model.list -> go to target folder -> read yaml -> return result)
[ ]  Model list (check model.list -> go to target folder -> read yaml -> return result)
[ ]  Start model/Run ()
[ ]  model update
[ ]  model delete
[ ]  Update gguf parser vs. yaml parser for new model.yaml template and add more inference parameters to support llamacpp

I'll create subtasks corresponding to above tasks. cc @dan-homebrew @0xSage @vansangpfiev @namchuai @louis-jan

Furthermore, given our previous issues with Model ID, can we clarify the relationship between:

model.list: model-id, model-alias fields
model.yaml: model field

For example, would this be how it works?
model.yaml[model] == model.list[model-alias]

I agree
model.list should look like this:
model-id author_repo-id branch-name path-to-model.yaml model-alias

And we should automatically create a short model alias and check for local duplication, add suffix if needed.

model-alias work exactly like model-id, with cortex command pull, run, update, delete, also with corresponding API. every action with model-id should also work with model-alias
model in model.yaml is used to load model locally. Cortex.llamacpp engine requires a field call model to identify what model user want to use (openAI API also require this field, so it's just an compatible api). So model.yaml[model] == model.list[model-alias] is great because it not only maintain the unique, but also short and human readable

@nguyenhoangthuan99 @louis-jan One more edge case to add. Can we make sure our Model Folder abstraction can work well with symlinks?
See: https://github.com/janhq/cortex.cpp/issues/1058
My quick thought is that I think it's just a symlink + model.yaml within the folder structure, but I'd like you guys to take a look too

Hi @dan-homebrew @nguyenhoangthuan99. I have mentioned it in the file protocol above, but it is not really clearly explained.
There are many reasons why we don't want to create a model symlink:

Model sources support the file protocol to refer to model files, model files can be located anywhere. We just create a model folder to structure imported model.yaml. E.g. models/imported/[name]/model.yaml.
Delete origin model could lead to a wrong file existing check since the symlink file still there.
Users might want to edit the file name or relocate files at any time. Recreating symlinks is harder than editing model.yaml.

model.yaml sources in this case could also be something like this:
sources:
   - C://path/to/model.gguf
   - C://path/to/model.bin

The import command can be effectively integrated into the current structure with the following specifications and considerations:
Command Syntax and Rationale
The command should follow the format: cortex models import <path> --model_id <model_id>. This structure is necessary because the model is being manually imported by the user, and the file name could be arbitrary (e.g., "anything.gguf"). Therefore, it is imperative to require the user to specify a model ID. The system will verify the uniqueness of the <model_id> and notify the user accordingly. By default,  will be set equal to .
Model Listing Format
The model.list entry will be structured as follows:
<model-id> local imported <path-to-model-id.yaml> <model-alias>
This format provides a clear and concise representation of the imported model's attributes.
Model Configuration File (model.yaml) Specification
For models imported from local files, the 'source' field in the model.yaml file should utilize the "files://" prefix instead of "models://". This distinction is crucial for differentiating between locally imported models and those from other sources.
Example of the sources section in model.yaml:
sources:             # Can be universal protocol (models://), absolute local file path (file://), or HTTPS remote URL (https://)
  - files://C:/Users/user/Downloads/Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf
  - files://C:/Users/user/Downloads/Mixtral-8x22B-v0.1-IQ3_M-00002-of-00005.gguf

This structure allows for clear differentiation between various source types (universal protocol, local file path, or remote URL) while maintaining consistency in the configuration file format.

Summary changes for model folder and model.yaml:
Model folder structure
/models
    model.list

    # Huggingface GGUF Model Folder format
    # This assumes we have some sort of quantization selection wizard when downloading?
    /huggingface.co
        /bartowski
            /Mixtral-8x22b-v0.1-gguf (includes all quants)
                Mixtral-8x22B-v0.1-IQ3_M.yaml
                Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf...
                Mixtral-8x22B-v0.1-Q8_M.yaml
                Mixtral-8x22B-v0.1-Q8_M-00001-of-00005.gguf...

    # Built-in Library Model Folder format
    /cortex.so (this is our Built-in Model Library, based on Git, that will be mirrored across a few sites)
        /llama3.1 (model)
            /q4-tensorrt-llm (tag)
                ...engine_files
                model.yaml
            /q8-gguf **(tag)**
                model.yaml
   # Imported model from local
    /imported
            <imported-model-id>.yaml

    # Future Model Source
    # Has its own model folder format

model.list
model-id author_repo-id branch-name path-to-model.yaml model-alias

for model imported by user:
<model-id> local imported <path-to-model-id.yaml> <model-alias>

And we should automatically create a short model alias and check for local duplication, add suffix if needed.

model-alias work exactly like model-id, with cortex command pull, run, update, delete, also with corresponding API. every action with model-id should also work with model-alias
model in model.yaml is used to load model locally. Cortex.llamacpp engine requires a field call model to identify what model user want to use (openAI API also require this field, so it's just an compatible api). So model.yaml[model] == model.list[model-alias] is great because it not only maintain the unique, but also short and human readable

model import command
The command should follow the format: cortex models import  --model_id <model_id>. This structure is necessary because the model is being manually imported by the user, and the file name could be arbitrary (e.g., "anything.gguf"). Therefore, it is imperative to require the user to specify a model ID. The system will verify the uniqueness of the <model_id> and notify the user accordingly. By default, <model_alias> will be set equal to <model_id>.
model.yaml
# BEGIN GENERAL GGUF METADATA
model: gemma-2-9b-it-Q8_0 # Model ID which is used for request construct - should be unique between models (author / quantization)
name: Llama 3.1      # metadata.general.name
version: 1           # metadata.version
sources:             # can be universal protocol (models://) OR absolute local file path (file://) OR https remote URL (https://)
  - models://huggingface/bartowski/Mixtral-8x22B-v0.1/main/Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf # for downloaded model from HF
  - files://C:/Users/user/Downloads/Mixtral-8x22B-v0.1-IQ3_M-00001-of-00005.gguf # for imported model
# END GENERAL GGUF METADATA

# BEGIN INFERENCE PARAMETERS
# BEGIN REQUIRED
stop:                # tokenizer.ggml.eos_token_id
  - <|end_of_text|>
  - <|eot_id|>
  - <|eom_id|>
# END REQUIRED
# BEGIN OPTIONAL
stream: true         # Default true?
top_p: 0.9           # Ranges: 0 to 1
temperature: 0.6     # Ranges: 0 to 1
frequency_penalty: 0 # Ranges: 0 to 1
presence_penalty: 0  # Ranges: 0 to 1
max_tokens: 8192     # Should be default to context length
# END OPTIONAL
# END INFERENCE PARAMETERS

# BEGIN MODEL LOAD PARAMETERS
# BEGIN REQUIRED
prompt_template: |+  # tokenizer.chat_template
  <|begin_of_text|><|start_header_id|>system<|end_header_id|>

  {system_message}<|eot_id|><|start_header_id|>user<|end_header_id|>

  {prompt}<|eot_id|><|start_header_id|>assistant<|end_header_id|>
# END REQUIRED
# BEGIN OPTIONAL
ctx_len: 0          # llama.context_length | 0 or undefined = loaded from model
ngl: 33             # Undefined = loaded from model
# END OPTIONAL
# END MODEL LOAD PARAMETERS

Break down to tasks:

[ ] Pull model (check model.list -> create folder -> download model -> create .yaml)
[ ] Model get (check model.list -> go to target folder -> read yaml -> return result)
[ ] Model list (check model.list -> go to target folder -> read yaml -> return result)
[ ] Start model/Run ()
[ ] model update
[ ] model delete
[ ] model alias
[ ] model import (import model from local)
[ ] Update gguf parser vs. yaml parser for new model.yaml template and add more inference parameters to support llamacpp

I'll create subtasks corresponding to above tasks. cc @dan-homebrew @0xSage  @vansangpfiev @namchuai @louis-jan

@nguyenhoangthuan99 I am shifting this to @vansangpfiev and tracking Tasklist items, just to keep big picture sitpic of progress.


Print an error on async stack overflow

This commit updates Wasmtime's handling of traps on Unix platforms to print an error message on stack overflow when the guard page is hit. This is distinct from stack overflow in WebAssembly which raises a normal trap and can be caught. This is instead to be used on misconfigured hosts where the async stack is too small or wasm was allowed to take up too much of the async stack. Currently no error message is printed and the program simply aborts with a core dump which can be difficult to debug.
This instead registers the range of the async guard page with the trap handling infrastructure to test the faulting address and if it lies within this range. If so then a small message is printed and then the program is aborted with libc::abort().
This does not impact the safety of any prior embedding or fix any issues. It's instead intended purely as a diagnostic tool to help users more quickly understand that stack size configuration settings are the likely culprit.

This is inspired by https://github.com/bytecodealliance/wasmtime/issues/9298 where stack overflows on the CLI in async fibers might be more common now.


Upgrading a service does not work as expected

We just deployed a new setup of rancher V1.6.18 running behind a nginx proxy with ssl termination.
Everything is working fine except when we try to upgrade a service.
Most of the time the progress starts spinning with "upgrading" and then it switches to "upgraded" but nothing actually happend. It did not stop or start any new containers or applied any of the changes to our settings. If we manually delete the containers after we finish the upgrade, everything will start up correctly with the new version and settings.
I checked the logs on the rancher server and can't see anything when upgrading a stack.
The weird thing is that sometimes it upgrades just fine and sometimes it just upgrades one of the containers and the rest when you press finish upgrade.
Our database is hosted on AWS but the server is running on a different host, could timeouts or delays to the database cause issues like this?

Useful
Info

Versions
Rancher v1.6.18 Cattle: v0.183.52 UI: v1.6.43

Access
github admin

Orchestration
Cattle

Route
service.containers

It also seems to work 70% of the time in chrome but safari will always just change status to upgraded without doing anything

Can you provide steps to reproduce, what stack definition and what steps are you taking? Does it happen for all stacks or just some? Since when did it start happening?

We have experienced this issue couple times too. =
One way to reproduce: If you try to delete containers several times during a service upgrade, the service will end up with state Upgraded but it actually did not. A workaround from us is to click finish upgrade button and the service would continue its journey to real "upgraded" state

Thanks for your report!
With the release of Rancher 2.0, development on v1.6 is only limited to critical bug fixes and security patches.
If you think we should keep this issue open, please let me know.


ufuncs for complex

Some ufuncs does not support complex types or behave differently compared with NumPy.

[x] rint (#1282)
[x] log1p, expm1, log2 (#1283)
[ ] logaddexp
[x] fmin, fmax (#1285)
[x] sign, square (#1284)

Hello, I would like to on work on logaddexp for complex numbers.

See https://github.com/cupy/cupy/pull/3113#issuecomment-591779449.
logaddexp does not support complex numbers.
>>> import numpy as np
>>> np.__version__
'1.18.1'
>>> np.logaddexp.types
['ee->e', 'ff->f', 'dd->d', 'gg->g']


"Parameter not valid" exception randomly occuring

Hi, I am trying to use your framework for recording automated tests. Unfortunately recording video is not stable - running and recording the same test produced different outcomes at random. For example, sometime the recording was complete, and at other times the video was only 10 seconds long (should be 30)
I've debugged your source code an the problem seems to be at:
Recorder.Record(), which catches exception "Parameter not valid" and stops the recording abruptly. Here is stacktrace:
at System.Drawing.Graphics.GetHdc()
at System.Drawing.Graphics.CopyFromScreen(Int32 sourceX, Int32 sourceY, Int32 destinationX, Int32 destinationY, Size blockRegionSize, CopyPixelOperation copyPixelOperation)
at System.Drawing.Graphics.CopyFromScreen(Point upperLeftSource, Point upperLeftDestination, Size blockRegionSize, CopyPixelOperation copyPixelOperation)
at Screna.ScreenShot.CaptureRegionManaged(Rectangle Region, Boolean IncludeCursor) in C:\Users\SG0222640\Desktop\Sample Projects\Screna-master\Screna-master\src\Core\ScreenShot.cs:line 277
at Screna.ScreenShot.Capture(Rectangle Region, Boolean IncludeCursor, Boolean Managed) in C:\Users\SG0222640\Desktop\Sample Projects\Screna-master\Screna-master\src\Core\ScreenShot.cs:line 297
at Screna.ScreenShot.Capture(Screen Screen, Boolean IncludeCursor, Boolean Managed) in C:\Users\SG0222640\Desktop\Sample Projects\Screna-master\Screna-master\src\Core\ScreenShot.cs:line 47
at Screna.ScreenProvider.OnCapture(Graphics g) in C:\Users\SG0222640\Desktop\Sample Projects\Screna-master\Screna-master\src\Core\ImageProviders\ScreenProvider.cs:line 32
at Screna.ImageProviderBase.Capture() in C:\Users\SG0222640\Desktop\Sample Projects\Screna-master\Screna-master\src\Core\ImageProviders\ImageProviderBase.cs:line 36
at Screna.Recorder.Record() in C:\Users\SG0222640\Desktop\Sample Projects\Screna-master\Screna-master\src\Core\Recorder\Recorder.cs:line 148
This exception is caught here:
catch (Exception e)
{
Stop();
RaiseRecordingStopped(e);
}
And the recording is stopped. Could you please look into it?
I am using AviCodec.X264

Please check the fixing-12 branch.

Thanks, Mathew, I will verify it on Monday at work and will let you know

OK

When will you release it on nuget

I have already released this on NuGet: v0.5.2

The fix is not working and the issue is still exist also RaiseRecordingStopped(e) doesn't call _recorder.RecordingStopped when this exception is occurred.
There is another strange issue I found and I guess this is the problem, the WriteFrameAsync Method doesn't write in the file until the recorder stopped manually, I mean the outout file is zero byte until the recording finished which lets the app take more much memory than it need until this error happens because of out of memory

I've also observed a similar problem before, but was not able to reproduce it after this fix.
However this may be linked with issue - Screna memory leaks + Video speed too fast #14

This is same issue but the update didn't solve it and I found the the image is fixed after some time in the recording

I check the application on another machine and it is working, I don't know why it is not working on my machine, probably it is caching issue
Thanks Mathew & Loydik for your help

There is another problem that I found WriteFrameAsync throw out of memory exception in Bitmap.LockBits

I solved this issue by re-implementing WriteFrameAsync by this code
` ///
/// Asynchronously writes an Image frame.
///
/// The Image frame to write.
/// The Task Object.
public Task WriteFrameAsync(Bitmap Image)
{
return Task.Run(async () =>
{
unsafe
{
var bits = Image.LockBits(new Rectangle(Point.Empty, Image.Size), ImageLockMode.ReadOnly, PixelFormat.Format32bppRgb);
//Marshal.Copy(bits.Scan0, _videoBuffer, 0, _videoBuffer.Length);
                byte* ImageStart = (byte*)bits.Scan0;
                Parallel.For(0, _videoBuffer.Length, y =>
                {
                    _videoBuffer[y] = ImageStart[y];
                });

                Image.UnlockBits(bits);
                Image.Dispose();
            }
            await _videoStream.WriteFrameAsync(true, _videoBuffer, 0, _videoBuffer.Length);
        });
    }`

I think the problem was that the Image was not being disposed which was leading to OutOfMemory error. I would prefer that the image is disposed from the recorder. Wait patiently for the next commit. Thank You.

@kroky-cs please check the latest commit.

Hi @MathewSachin the same problem.
your solution will be the best working solution if you implement it not in recorder but before this line
await _videoStream.WriteFrameAsync(true, _videoBuffer, 0, _videoBuffer.Length);
in the public Task WriteFrameAsync(Bitmap Image) in the AviWriter class
`public Task WriteFrameAsync(Bitmap Image)
{
return Task.Run(async () =>
{
var bits = Image.LockBits(new Rectangle(Point.Empty, Image.Size), ImageLockMode.ReadOnly, PixelFormat.Format32bppRgb);
Marshal.Copy(bits.Scan0, _videoBuffer, 0, _videoBuffer.Length);
Image.UnlockBits(bits);
Image.Dispose();
            await _videoStream.WriteFrameAsync(true, _videoBuffer, 0, _videoBuffer.Length);
        });

}`

Check the latest commit. Did exactly what you said.

@MathewSachin Thanks alot, you are my man
If you have time can you prepare nuget version?

I'm sorry, you'll have to wait for 3 hrs. As soon as I get home I would do that

no problem, I am not in a hurry.

Released v0.5.3

Thanks @MathewSachin

Thanks & Regards
[image: Venta Apps] http://www.ventaapps.com/
Karim Abd El-AAl / Co-founder
<EMAIL_ADDRESS>/ +201000958778
Venta Apps
<EMAIL_ADDRESS>/ +2035750078
6 Zo El Fakar St. Loran, Alexandria, Egypt
http://www.ventaapps.com
[image: Facebook]  https://www.facebook.com/krokys[image: Twitter]
https://twitter.com/Kroky_cs[image: Google Plus]
https://plus.google.com/+KarimAbdElAAl[image: Linkedin]
https://www.linkedin.com/in/kroky[image: skype]
This e-mail message may contain confidential or legally privileged
information and is intended only for the use of the intended recipient(s).
Any unauthorized disclosure, dissemination, distribution, copying or the
taking of any action in reliance on the information herein is prohibited.
E-mails are not secure and cannot be guaranteed to be error free as they
can be intercepted, amended, or contain viruses. Anyone who communicates
with us by e-mail is deemed to have accepted these risks. Venta Apps is not
responsible for errors or omissions in this message and denies any
responsibility for any damage arising from the use of e-mail. Any opinion
and other statement contained in this message and any attachment are solely
those of the author and do not necessarily represent those of the company.
On Tue, Oct 25, 2016 at 5:08 PM, Mathew Sachin<EMAIL_ADDRESS>wrote:

Released v0.5.3
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/MathewSachin/Screna/issues/12#issuecomment-256063017,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AF5NgcDA_FbVVQQFyJbYmcoGIFC1bLV7ks5q3htpgaJpZM4KXEDd
.


Remove version: taers232c.GAMADV-XTD3 version 6.79.00

This version is older than what has been set in max-versions-to-keep by the publisher.
Microsoft Reviewers: Open in CodeFlow

URL: https://github.com/taers232c/GAMADV-XTD3/releases/download/v6.79.00/gamadv-xtd3-6.79.00-windows-x86_64.msi
OK
(Automated message - build 913)


Release for AGX Orin Devkit requested

This is needed for shipping https://github.com/balena-os/balena-jetson-orin/commit/2d3f73c04c3073d220d906a9c4591e5b7c535a8c to production.
The AGX Orin Devkit is in the queue for Autokit, currently the Orin Nano is in progress of being added.

Now available in balena-cloud at v4.15.1+rev1


Connection resets from sidecar when deleting a pod

Bug Description
We are trying to achieve zero downtime deployment but we are getting some 502 http errors the moment a pod is being deleted.
We have the following setup:

Public ALB on instance mode (we are facing the issue when using NLB as well)
Cilium CNI (datapath: encapsulation/ipam: cluster pool/tunnel: vxlan)
istio-proxy sidecar version 1.14.1
externalTrafficPolicy: Cluster on the respective service

The backend application supports graceful shutdown and we have added a preStop hook both on the application and the istio sidecar but the issue is that whenever the pods gets into terminating phase - while the SIGTERM signal has not yet reached the containers due to the preStop hook - we are getting connection reset causing the ALB to return 502 http errors.
Note that we are not getting any error when removing the istio-proxy sidecar.
Here is an example flow:

at 16:47:41 (UTC+3) we are deleting a pod (poltest-server-74ff5987c9-66fwp)
The moment we are deleting the pod we are getting a lot of the following error from the load test tool we are using (fortio):

16:47:41 W http_client.go:838> [8] Non ok http code 502 (HTTP/1.1 502)

istio-proxy receives the SIGTERM signal 4 seconds later (due to the preStop hook). Relevant logs (level: info, UTC time):

2022-08-10T13:47:45.235633Z	info	Status server has successfully terminated
2022-08-10T13:47:45.235635Z	info	Agent draining Proxy
2022-08-10T13:47:45.235730Z	error	accept tcp [::]:15020: use of closed network connection
2022-08-10T13:47:45.236622Z	info	Graceful termination period is 5s, starting...
2022-08-10T13:47:46.698820Z	warn	Envoy proxy is NOT ready: server is not live, current state is: DRAINING
2022-08-10T13:47:48.699140Z	warn	Envoy proxy is NOT ready: server is not live, current state is: DRAINING
2022-08-10T13:47:50.237764Z	info	Graceful termination period complete, terminating remaining proxies.
2022-08-10T13:47:50.237815Z	warn	Aborted all epochs
2022-08-10T13:47:50.237823Z	warn	Aborting epoch 0
2022-08-10T13:47:50.237934Z	info	Epoch 0 aborted normally
2022-08-10T13:47:50.238018Z	info	Agent has successfully terminated

using tcpdump (capturing the traffic on the relevant instance nodeport 32511) and wireshark we are getting RST packet:

Version
$ istioctl version
client version: 1.14.1
control plane version: 1.14.1
data plane version: 1.14.1 (93 proxies)

$ kubectl version --short
Client Version: v1.21.0
Server Version: v1.21.7

This is ALB -> Pod (with sidecar) directly right? No ingressgateway?
Are you able to test going to pod directly (not via ALB) and/or removing the sidecar from the pod and see if it reproduces?

Right, ALB --> Pod (with sidecar). Actually, it is alb target group --> instance nodeport (so there can be another extra hop from instance to instance).
We have tested also with NLB (no ingressgateway there as well) and we are getting those connection resets as well.
On the other hand, there are no errors when testing with ELB (ELB --> nginx proxy pod --> application pod) and when testing locally by port-forwarding the service (sidecar enabled in both cases).

port-forwarding to the service bypasses the sidecar, can you try hitting thecluster IP directly? That would rule out any AWS LB specific things

Test using cluster IP (no errors):
Testing by hitting the cluster IP we receiving no errors. Results:
fortio load -c 8 -qps 300 -t 1m -timeout 3s http://<IP_ADDRESS>:8088/ping

Fortio 1.33.0 running at 300 queries per second, 8->8 procs, for 1m0s: http://<IP_ADDRESS>:8088/ping
15:13:07 I httprunner.go:93> Starting http test for http://<IP_ADDRESS>:8088/ping with 8 threads at 300.0 qps and parallel warmup
Starting at 300 qps with 8 thread(s) [gomax 8] for 1m0s : 2250 calls each (total 18000)
15:14:07 I periodic.go:723> T007 ended after 1m0.002872392s : 2250 calls. qps=37.498204840940005
15:14:07 I periodic.go:723> T006 ended after 1m0.002881974s : 2250 calls. qps=37.498198852764325
15:14:07 I periodic.go:723> T004 ended after 1m0.002949746s : 2250 calls. qps=37.498156499380976
15:14:07 I periodic.go:723> T002 ended after 1m0.002956631s : 2250 calls. qps=37.49815219667954
15:14:07 I periodic.go:723> T001 ended after 1m0.002969738s : 2250 calls. qps=37.49814400561362
15:14:07 I periodic.go:723> T005 ended after 1m0.003091697s : 2250 calls. qps=37.49806778893852
15:14:07 I periodic.go:723> T003 ended after 1m0.003148185s : 2250 calls. qps=37.49803248760988
15:14:07 I periodic.go:723> T000 ended after 1m0.003167124s : 2250 calls. qps=37.498020651980674
Ended after 1m0.003192912s : 18000 calls. qps=299.98
Sleep times : count 17992 avg 0.024014967 +/- 0.0007295 min 0.00570817 max 0.025396916 sum 432.077287
Aggregated Function Time : count 18000 avg 0.0020312857 +/- 0.0006166 min 0.00111114 max 0.020600165 sum 36.5631434
# range, mid point, percentile, count
>= 0.00111114 <= 0.002 , 0.00155557 , 54.11, 9739
> 0.002 <= 0.003 , 0.0025 , 98.80, 8045
> 0.003 <= 0.004 , 0.0035 , 99.40, 108
> 0.004 <= 0.005 , 0.0045 , 99.52, 22
> 0.005 <= 0.006 , 0.0055 , 99.61, 16
> 0.006 <= 0.007 , 0.0065 , 99.73, 21
> 0.007 <= 0.008 , 0.0075 , 99.79, 12
> 0.008 <= 0.009 , 0.0085 , 99.89, 18
> 0.009 <= 0.01 , 0.0095 , 99.91, 3
> 0.01 <= 0.011 , 0.0105 , 99.92, 1
> 0.011 <= 0.012 , 0.0115 , 99.94, 4
> 0.012 <= 0.014 , 0.013 , 99.98, 7
> 0.016 <= 0.018 , 0.017 , 99.98, 1
> 0.02 <= 0.0206002 , 0.0203001 , 100.00, 3
# target 50% 0.00193255
# target 75% 0.0024675
# target 90% 0.00280311
# target 99% 0.00333333
# target 99.9% 0.00933333
Error cases : no data
15:14:07 I httprunner.go:190> [0] <IP_ADDRESS>:8088 resolved to <IP_ADDRESS>:8088
15:14:07 I httprunner.go:190> [1] <IP_ADDRESS>:8088 resolved to <IP_ADDRESS>:8088
15:14:07 I httprunner.go:190> [2] <IP_ADDRESS>:8088 resolved to <IP_ADDRESS>:8088
15:14:07 I httprunner.go:190> [3] <IP_ADDRESS>:8088 resolved to <IP_ADDRESS>:8088
15:14:07 I httprunner.go:190> [4] <IP_ADDRESS>:8088 resolved to <IP_ADDRESS>:8088
15:14:07 I httprunner.go:190> [5] <IP_ADDRESS>:8088 resolved to <IP_ADDRESS>:8088
15:14:07 I httprunner.go:190> [6] <IP_ADDRESS>:8088 resolved to <IP_ADDRESS>:8088
15:14:07 I httprunner.go:190> [7] <IP_ADDRESS>:8088 resolved to <IP_ADDRESS>:8088
Sockets used: 8 (for perfect keepalive, would be 8)
Uniform: false, Jitter: false
IP addresses distribution:
<IP_ADDRESS>:8088: 8
Code 200 : 18000 (100.0 %)
Response Header Sizes : count 18000 avg 172.00078 +/- 0.02788 min 172 max 173 sum 3096014
Response Body/Total Sizes : count 18000 avg 190.00078 +/- 0.02788 min 190 max 191 sum 3420014
All done 18000 calls (plus 8 warmup) 2.031 ms avg, 300.0 qps

Test using Nodeport(we get connection resets):
Testing using a random worker IP and the respective service nodeport we have the following results:
fortio load -c 8 -qps 300 -t 1m -timeout 3s http://<IP_ADDRESS>:32511/ping

Fortio 1.33.0 running at 300 queries per second, 8->8 procs, for 1m0s: http://<IP_ADDRESS>:32511/ping
15:18:14 I httprunner.go:93> Starting http test for http://<IP_ADDRESS>:32511/ping with 8 threads at 300.0 qps and parallel warmup
Starting at 300 qps with 8 thread(s) [gomax 8] for 1m0s : 2250 calls each (total 18000)
15:18:27 I http_client.go:806> [6] Closing dead socket <IP_ADDRESS>:32511 (err read tcp <IP_ADDRESS>:47814-><IP_ADDRESS>:32511: read: connection reset by peer at first read)
15:18:27 I http_client.go:806> [1] Closing dead socket <IP_ADDRESS>:32511 (err read tcp <IP_ADDRESS>:47824-><IP_ADDRESS>:32511: read: connection reset by peer at first read)
15:19:14 I periodic.go:723> T005 ended after 1m0.001684947s : 2250 calls. qps=37.49894693769757
15:19:14 I periodic.go:723> T000 ended after 1m0.001825608s : 2250 calls. qps=37.49885902971608
15:19:14 I periodic.go:723> T007 ended after 1m0.001830348s : 2250 calls. qps=37.49885606739658
15:19:14 I periodic.go:723> T004 ended after 1m0.001858883s : 2250 calls. qps=37.49883823411812
15:19:14 I periodic.go:723> T006 ended after 1m0.001844539s : 2250 calls. qps=37.49884719856479
15:19:14 I periodic.go:723> T003 ended after 1m0.001895781s : 2250 calls. qps=37.49881517431117
15:19:14 I periodic.go:723> T002 ended after 1m0.001907321s : 2250 calls. qps=37.49880796226831
15:19:14 I periodic.go:723> T001 ended after 1m0.001930662s : 2250 calls. qps=37.49879337507642
Ended after 1m0.001948527s : 18000 calls. qps=299.99
Sleep times : count 17992 avg 0.024508046 +/- 0.0006779 min 0.012145583 max 0.025605314 sum 440.948767
Aggregated Function Time : count 18000 avg 0.0015282839 +/- 0.0005055 min 0.000883541 max 0.014081757 sum 27.5091095
# range, mid point, percentile, count
>= 0.000883541 <= 0.001 , 0.00094177 , 0.02, 4
> 0.001 <= 0.002 , 0.0015 , 94.96, 17088
> 0.002 <= 0.003 , 0.0025 , 98.97, 722
> 0.003 <= 0.004 , 0.0035 , 99.43, 84
> 0.004 <= 0.005 , 0.0045 , 99.56, 23
> 0.005 <= 0.006 , 0.0055 , 99.65, 16
> 0.006 <= 0.007 , 0.0065 , 99.78, 24
> 0.007 <= 0.008 , 0.0075 , 99.86, 14
> 0.008 <= 0.009 , 0.0085 , 99.88, 3
> 0.009 <= 0.01 , 0.0095 , 99.96, 14
> 0.01 <= 0.011 , 0.0105 , 99.99, 6
> 0.012 <= 0.014 , 0.013 , 99.99, 1
> 0.014 <= 0.0140818 , 0.0140409 , 100.00, 1
# target 50% 0.00152645
# target 75% 0.00178979
# target 90% 0.0019478
# target 99% 0.00307143
# target 99.9% 0.00928571
Error cases : no data
15:19:14 I httprunner.go:190> [0] <IP_ADDRESS>:32511 resolved to <IP_ADDRESS>:32511
15:19:14 I httprunner.go:190> [1] <IP_ADDRESS>:32511 resolved to <IP_ADDRESS>:32511
15:19:14 I httprunner.go:190> [2] <IP_ADDRESS>:32511 resolved to <IP_ADDRESS>:32511
15:19:14 I httprunner.go:190> [3] <IP_ADDRESS>:32511 resolved to <IP_ADDRESS>:32511
15:19:14 I httprunner.go:190> [4] <IP_ADDRESS>:32511 resolved to <IP_ADDRESS>:32511
15:19:14 I httprunner.go:190> [5] <IP_ADDRESS>:32511 resolved to <IP_ADDRESS>:32511
15:19:14 I httprunner.go:190> [6] <IP_ADDRESS>:32511 resolved to <IP_ADDRESS>:32511
15:19:14 I httprunner.go:190> [7] <IP_ADDRESS>:32511 resolved to <IP_ADDRESS>:32511
Sockets used: 10 (for perfect keepalive, would be 8)
Uniform: false, Jitter: false
IP addresses distribution:
<IP_ADDRESS>:32511: 8
Code 200 : 18000 (100.0 %)
Response Header Sizes : count 18000 avg 259 +/- 0 min 259 max 259 sum 4662000
Response Body/Total Sizes : count 18000 avg 277 +/- 0 min 277 max 277 sum 4986000
All done 18000 calls (plus 8 warmup) 1.528 ms avg, 300.0 qps

Checking the results fortio reports 100.0% success code (http 200), so it seems that in this case the connection resets do not drop any inflight request. (in the case of ALB those connection resets are causing a lot of 502 http errors)
15:18:27 is the time we deleted a pod.
For the record, here is the service details:
$ kubectl describe svc poltest-server
Name:                     poltest-server
Namespace:                polykarpos
Labels:                   app=poltest-server
                          app.kubernetes.io/instance=poltest-server
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=poltest-server
                          domain=platform
                          helm.sh/chart=poltest-server
                          team=exampleTeam
                          version=1.0
Annotations:              meta.helm.sh/release-name: poltest-server
                          meta.helm.sh/release-namespace: polykarpos
Selector:                 app.kubernetes.io/instance=poltest-server,app.kubernetes.io/name=poltest-server
Type:                     NodePort
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       <IP_ADDRESS>
IPs:                      <IP_ADDRESS>
Port:                     http  8088/TCP
TargetPort:               8088/TCP
NodePort:                 http  32511/TCP
Endpoints:                <IP_ADDRESS>:8088,<IP_ADDRESS>:8088,<IP_ADDRESS>:8088
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

To my understanding, the moment a pod goes into terminating state, k8s endpoint controller removes the respective pod ip from the service's endpoints without dropping any connection, since both the proxy sidecar and the application are still alive due to the preStop hook. And in the above tests fortio is using keepalives so we should not observe the above connection resets.
I do not know the istio internals, so I was wondering if istiod monitors the service's endpoints list and performs any action following a change on those endpoints.


"os" in package.json is causing issues in webpack build

When using this package, my CI is failing due to this error
 npm ERR! notsup Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"win32","arch":"any"} (current: {"os":"linux","arch":"x64"})

so I installed native-reg as an optional dependency in order to skip it if the OS is not Windows.
Now I get this error when webpack builds the app
Module not found: Error: Can't resolve 'native-reg'

I couldn't find a good solution to this issue
To fix this, I'm using my fork right now in which I removed the os property from package.json

What a nightmare! So, webpack actually supports several ways to handle optional dependencies, but it's not actually optional for you.
There are currently a few ways you can get npm to install an os specific package, but most of them seem like bugs. --force will, but will also change a whole bunch of other behavior you don't want. There is an --ignore-engines, but no --ignore-os. There doesn't seem to be any sensible other option here other than just remove the os field which seems basically unsupported: but now npm will try and fail to build it from source on your CI. I would have expected that to have failed the install, but even if it's not, that should also be fixed.

When removing the os field the CI doesn't fail :)

If you change your code to
let nativeReg;
if (process.platform === 'win32') {
  nativeReg = require('native-reg');
}

if (nativeReg) {
  // ...
}

Would that remove the Module not found: Error: Can't resolve 'native-reg' message?

@TheJaredWilcurt If I remember correctly this was not enough for me. I think the build still fails unless I put native-reg as an optional dependency.
I've been doing what you've suggested but with my fork "@aabuhijleh/native-reg" in which I removed "os" from package.json
Nevertheless, this solution isn't pretty especially when you're using typescript where it's really preferred to use the import syntax, not require.
I'm not sure but I think the best solution is to make a no-op version of this module for other platforms besides Windows.
Similar to what this native module is doing.


Swap USWDS CDN for USWDS-Gulp

Currently 11ty is pulling in UWDS via CDN. Swap it to build with uswds-gulp.
uswds-gulp allows us to:

Customize what parts of UWDS we want to include in the app
Use custom variables/etc with USWDS components
Use USWDS variables/tokens/etc with custom styles

What's the status for this?


Pagination bug

When there isn't a "/" at the end of the URL the first page of results is missing the full URL in the pagination links.

Closing in lieu of new setup


SCK rate

Hi, I've seen in readme that changing SCK rate is not implemented yet. But can you give me a clue how to achieve it with your app? I went through the source code of sht-sensor and couldn't figure it out.
Thanks

The reason I'm thinking of lowering SCK, because I have pretty long cable (about 20m) and I get an error "Command ACK failed on step-2". Maybe I should look elsewhere to solve this problem, any hints?

Hi,
There isn't much in the code that controls that rate, as it should be kinda slow anyway, with bit-banging done from python through sysfs gpio interface.
But to make it consistent and then set to something you want, best place to do it should be "_sck_tick" method here:
https://github.com/mk-fg/sht-sensor/blob/master/sht_sensor/sensor.py#L133-L135
It's used every time when clock must go 0->1 or 1->0, with example right below linked lines.
To set an upper limit on how fast clock can tick, you can simply put higher value into sleep() function there - I suspect sensor logic won't care even if it'd be seconds (as it doesn't have its own clock to know).
Or, to make it somewhat more consistent, I'd suggest to:

Use monotonic clock - e.g. using python 2.7 code from here: https://github.com/mk-fg/python-pulse-control/blob/master/pulsectl/_pulsectl.py#L30-L42
Python 3.x has this func as time.monotonic(), but 2.x only has time.time(), which should not be used for such purposes.

On every call to "_sck_tick", get last tick and current (monotonic clock) timestamp, sleep until "last_ts + tick_interval" if it's still in the future, store current timestamp (either post-sleep or pre-sleep + delay) as "last_ts".

That way, you shouldn't have any extra delays when ticks are separated in time far enough already.
Though making a syscall to get monotonic timestamp through python ctypes interface with all the associated overhead might cause enough delay as it is, without any sleep :)
Don't really have any back-of-the-napkin numbers on that however, and these should depend heavily on cpu and system load anyway.
Might implement it here eventually, should be rather trivial, but need to find/hook-up sensor to something for testing, and need to attend to other stuff at the moment.

Might implement it here eventually, should be rather trivial, but need to find/hook-up sensor to something for testing

Done.
More info on how to specify it here:
https://github.com/mk-fg/sht-sensor/#max-bit-banging-frequency-control
Datasheet indeed says that min frequency is 0, i.e. "take as long as you want" ;)
Testing with Beaglebone Black though, values as low as 20Hz (50ms delay between changes) work reliably, but 10Hz doesn't... wonder why.
Fairly sure you want higher values anyway though.
Let me know if it works for you, or maybe has any issues.

wow, thanks for quick update! appreciate it
Although, in my case, it seems its not the long cable issue. I have tried with short cable again (~5cm) and I still get "Command ACK failed on step-2". Can you please describe possible reasons for this error?
I tried several variations connecting the sensor, with different GPIO pins, and also tried sht1x lib you reference in readme. The same problem. I've tested with oscilloscope and SCK signal is quite strong, but DAT is silent. I assume I could screw up my sensor while soldering back'n'forth to different wires, since SHT75 is quite gentle (comparing to SHT25, for example). So I just ordered new one just to make sure, waiting for it arrive.
I will try updated frequency control for sure, I'll post an update here when new sensor arrives. thanks

still get "Command ACK failed on step-2". Can you please describe possible reasons for this error?

After sending preamble and command the the sensor, DATA line goes into IN mode and following sequence should be observed:

SCK=0 (after last bit of command)
SCK=1
DATA=1 ("ACK step 1" from sensor)
SCK=0
DATA=0 ("ACK step 2" from sensor)

That way sensor should acknowledge that it has received the command, according to the datasheet.
Reading e.g. temperature then is done by sending "get temperature" command (with that ACK sequence checked at the end), followed by reading value + value + crc bits from sensor, each separated by similar ACK sequence sent to the sensor (indicating that we want second copy of value and crc)... iirc.

tested with oscilloscope and SCK signal is quite strong, but DAT is silent

If you don't see signals from the board there, then maybe it's the connection after all?
Might also be sensor pulling the value up/down too strongly, I guess, so that setting 1/0 on that kind of short-circuited pin doesn't make a difference, but that seem unlikely, and probably easy to rule-out by disconnecting the sensor temporarily and listening for command bits there.

Hm, right, with sensor soldered to wires like that, guess it might be not as trivial to disconnect it as I made it sound above, except maybe cutting wires and re-soldering them later...

Also, as a side note, SHT75 fits well into 1.27mm-step sockets, e.g.:
https://www.aliexpress.com/item/100-pcs-0-050-2x4-P-8-Pin-1-27-mm-Male-Female-SMT-Female-DIP/32769141515.html
And these should be always safe to solder without risking any damage to the (unplugged) sensor.


Add support for links with hashes

Hi!
This might be possible already, but it does not seem to be working for me.
I have the following:
active_link_to "Emissions", city_path(city, anchor: "#city-emissions") but the presence of the hash is not triggering the link activation.
I have tired active: :inclusive, active: :exclusive, and active: :exact.
Any tips?
Thanks

Just released v.1.0.4 and I added test that seems to work for your issue: https://github.com/comfy/active_link_to/blob/master/test/active_link_to_test.rb#L138
Please reopen if this is still an issue. Better yet, create a failing test.


Why can a string of type any be assigned to type number?

let str: any= 'str' let num: number = str //ok

Because this is the reason of the type any: be assignable from and to any other type. See it on basic types. Recommendation, read the FAQs.
Better ask questions to stackoverflow.


FPS unlocker not working after updating to .NET 8.0.0 on GNU/Linux

OS specifications:
Arch Linux
6.6.8-arch1-1
i5-11400F
RX 570
16 GiB
FPS unlocker is  opening with an error to install .NET 8.0.0, even after already installed requirements.
Is there any solution for this problem? Thanks in advance.
Screenshot of the problem:

You'll probably need to ask the linux community instead


First Missing Positive - LeetCode

Link of the question
https://leetcode.com/problems/first-missing-positive/
Coding platform

[x] LeetCode

Programming language

[x] Java

Additional context (OPTIONAL)

Kindly assign me this issue @avinash201199 under Hacktoberfest'21.


Upgrade workflows to gradle/gradle-build-action@v2

Upgrades the GitHub actions workflows to use gradle/gradle-build-action@v2 (presently v2.1).
This action implementation replaces the previous eskatos/gradle-command-action@v1.
This new action:

has simpler configuration
does a better job of caching the Gradle User Home state between build invocations
automatically adds annotations with build scan URLs for all gradle invocations.

Due to the second item, I've removed the separate use of actions/cache for caching.
I've also removed the "Stop Gradle Daemon" steps, since this is handled by the action. Currently, this is done with org.gradle.daemon=false, but an upcoming version will allow the daemon to run between steps and stop at the end of the workflow.

@CsCherrYY This PR replaces #1047

@bigdaz Thank you! I'll approve and merge this PR.


Allow lookup template attributes from globalTemplateAttributes

This PR allow xcodegen to lookup template attributes from globalTemplateAttributes spec to share attributes accross all yaml files.
For example here, we have a shared today-extension-target.yml between multi project specs: legacy-project.yml and new-project.yml
We need to configure different config files between legacy and old. With this PR we just declare a version attribute from project spec and today-extension-target.yml can access it
Example
today-extension-target.yml
targetTemplates:
  TodayExtTemplate:
    platform: iOS
    type: app-extension
    settings:
      CODE_SIGN_IDENTITY: Apple Development
    configFiles:
      Debug: Config/${version}/Debug.xcconfig
      Release: Config/${version}/Release.xcconfig
    sources:
      - TodayExtension

targets:
  TodayExt:
    templates:
      - TodayExtTemplate

legacy-project.yml
name: Demo

options:
  createIntermediateGroups: True

globalTemplateAttributes:
  version: legacy

include:
  - today-extension-target.yml

targets:
  Demo:
    type: application
    platform: iOS
    deploymentTarget: "10.0"
    sources:
      - DemoXcodeGenGlobalTemplateAttribute

new-project.yml
name: Demo

options:
  createIntermediateGroups: True

globalTemplateAttributes:
  version: new

include:
  - today-extension-target.yml

targets:
  Demo:
    type: application
    platform: iOS
    deploymentTarget: "10.0"
    sources:
      - DemoXcodeGenGlobalTemplateAttribute

@yonaskolb Please help me review this PR

@yonaskolb Please help me review this PR. Thanks

@yonaskolb Please help me review this PR. Thanks

I can see the utility in this @vikage. Got a few notes:

does the existing functionality of referencing environment variables in the spec also solve your issue? eg Config/${VERSION}/Debug.xcconfig and VERSION=new xcodegen generate
If this were added I can see this being useful just as globally accessible values, as opposed to restricting to just target or scheme templates. What do you think?
If this were moved forward, it would definitely need some unit tests, and an example in Tests/Fixtures/TestProject/project.yml

I can see the utility in this @vikage. Got a few notes:

does the existing functionality of referencing environment variables in the spec also solve your issue? eg Config/${VERSION}/Debug.xcconfig and VERSION=new xcodegen generate
If this were added I can see this being useful just as globally accessible values, as opposed to restricting to just target or scheme templates. What do you think?
If this were moved forward, it would definitely need some unit tests, and an example in Tests/Fixtures/TestProject/project.yml

@yonaskolb

Currently, exporting a variable on the command line could fix my issue. But I think it will be great if we have it in the yml config file.
If this was added, we could access variables globally without any restrictions.

Yes happy for you to move it forward, with the ability to reference these values anywhere and not just within templates

@yonaskolb Please help me take a look

@yonaskolb Please review, thanks


disconnect if selection of db fails

tornadis always uses the default db0 after redis-server restarts regardless of the db number which was set during tornadis client initialization. This occurs when redis-server version is lower than 2.6.
"select" operation doesn't work when redis is loading the dataset. https://github.com/antirez/redis/issues/1024

Coverage increased (+0.01%) to 90.748% when pulling 4a252258cd9195fbab8be4893c289736de57fb82 on alex-67:select-db-fix into f2cf6306bf0e35175d74585ca36f93d56464b958 on thefab:master.


Fix instructions for generating query graph

The original instructions at the bottom of this page don't work for me.
I'm getting error saying that generate_querygraph.py doesn't exist.
I looked into my local installation of duckdb within
/home/jhrcek/.local/lib/python3.10/site-packages/duckdb and I didn't find any file named generate_querygraph.py
Instead there's this (running ls in the above folder):
├── query_graph
│   ├── __main__.py
│   └── __pycache__
│       └── __main__.cpython-310.pyc

When executed using python -m duckdb.query_graph /path/to/file.json it works for me (generates html file and opens it in my browser).

This was outdated yeah, thanks for the fix!


Specific nuget.exe version and nuspec mac fixes

Hi @Redth, I'm your friendly neighborhood .NET Foundation Pull Request Bot (You can call me DNFBOT). Thanks for your contribution!
You've already signed the contribution license agreement. Thanks!
The agreement was validated by .NET Foundation and real humans are currently evaluating your PR.
TTYL, DNFBOT;


Question: Use Local Storage (client-side) for chat persistence to enable multi-chat functionality without a backend database

Dear all,
I was wondering if there are any ways or plans to enable multi-chat functionalities without requiring the persistence of conversations in a database?
We are not permitted to persist any conversations in our database, and the conversations must be treated as confidential. Therefore, I wanted to ask if you have any thoughts on how to make this work?

Not at this moment, but we're open to suggestions as to how you imagine this to be implemented (architecture-wise). If we can reach consensus on the how, and it's automatically tested (so our support load doesn't increase much) this might be a great feature to have!
A good first step would be to abstract storage interaction in the frontend - see where and whether you can inject hooks there that catch things before going to the server.


EVERST-1140 | Add CLI version requirements for 0.11.0

Test name
Status

api-tests
passed

Closing in favour of https://github.com/Percona-Lab/percona-version-service/pull/242


pull request for e2e test at 01/20/2017 04:56:18

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Observable.create() vs Observable.fromAsync()

RxJava 1.1.7 introduced Observable.fromAsync() to "... bridge the callback world with the reactive". Would implementing Observable.fromAsync() with the callback-based view listeners in Android be functionally the same as the Observable.create() with Observable.OnSubscribe<T> method that RxBinding uses?
Example Observable.fromAsync() vs TextViewTextOnSubscribe
public static Observable<CharSequence> textChanges(final TextView textView) {
        return Observable.fromAsync(new Action1<AsyncEmitter<CharSequence>>() {
            @Override
            public void call(final AsyncEmitter<CharSequence> emitter) {
                final TextWatcher textWatcher = new SimpleTextWatcher() {
                    @Override
                    public void afterTextChanged(Editable s) {
                        emitter.onNext(s);
                    }
                };
                textView.addTextChangedListener(textWatcher);

                final AsyncEmitter.Cancellable cancellable = new AsyncEmitter.Cancellable() {
                    @Override
                    public void cancel() throws Exception {
                        textView.removeTextChangedListener(textWatcher);
                    }
                };
                emitter.setCancellation(cancellable);
            }
        }, AsyncEmitter.BackpressureMode.BUFFER); // not quite sure what BackpressureMode would be acceptable
    }

Are there potential performance or output differences between the two implementations? Observable.fromAsnyc() is still flagged as Experimental as well.
(was not sure whether to post this question to RxJava or RxBinding -- will move if this is out of place)

Would implementing Observable.fromAsync() with the callback-based view listeners in Android be functionally the same as the Observable.create() with Observable.OnSubscribe<T> method that RxBinding uses?

Yes. As a library we cannot use it because it's not a stable API though. It's great for use in applications though.

Are there potential performance or output differences between the two implementations?

No, they will be exactly the same in performance and function.

If this gets promoted to the regulären api: which backpressure Mode would be suitable for this library?

I would use NONE in the library. It's the user's choice (via applying onBackpressureXxx) if they want to add backpressure. By default, though, I wouldn't include any (which is what we do now with create()).


Deno Upgrade broke my apps

Upgraded to Deno 1.30.0 and now my apps fail with
error: Uncaught TypeError: Cannot read properties of undefined (reading 'ops')
const DECODED = Deno.build.os === "windows" && Deno.build.arch === "x86_64"
? Deno.core.ops.op_base64_decode(BASE64) : new Uint8Array();

            ^
    at https://glfw-binaries.deno.dev/3.4.0/glfw3_windows.js:2:89

I downloaded a fresh repo of dwm, and tried the examples! Same error!
\C:\Users\nhron\dev\GitHub\dwm> cd examples
\C:\Users\nhron\dev\GitHub\dwm\examples> deno run -A --unstable canvas.ts
error: Uncaught TypeError: Cannot read properties of undefined (reading 'ops')
const DECODED = Deno.build.os === "windows" && Deno.build.arch === "x86_64" ? Deno.core.ops.op_base64_decode(BASE64) : new Uint8Array();
                                                                                ^
    at https://glfw-binaries.deno.dev/3.4.0/glfw3_windows.js:2:89
\C:\Users\nhron\dev\GitHub\dwm\examples>

upgrade off of canary, we're fixing this issue should have this issue fixed asap

Not on canary!  I upgraded to the not current version Deno V1.30.0

will update asap

It's blocked on new release of Skia Canvas. Will update it and cut DWM release as soon as possible.

New releases are out:

DWM: v0.3.0
Skia Canvas: v0.5.1

Both should work on Deno v1.30.0.


Add SNS publish SMS test

Added test for SNS publish SMS attributes. Tests to support #236

Awesome, is this good to go?

¯_(ツ)_/¯ Tests pass so if you're happy with it, sure.


[Design Tweak] If no information on reserve or max bid (smaller text below 'starting bid') move the rest of the content up

Before:

After:

Fails

:no_entry_sign:
No CHANGELOG added.

  Generated by :no_entry_sign: dangerJS


Fix for app not covering full screen

before:

now:

also
Added prettier config file so that everyone's code formatter will behave same.
and doesn't produce useless changes into files without any reason.
I haven't changed any code just format it once thats why there are so many changes.
no need to worry about any breaking changes in code.


[tflite] Update dependency from tflite to tf2lite

Update dependency from tensorflow-lite to tensorflow2-lite.
This is done for Tizen and Debian build. It has already been done for Android.
Tensorflow2-lite header is dependent on flatbuffer, so flatbuffers-devel
is additionally added as a build-dependency.
Self evaluation:

Build test: [x]Passed [ ]Failed [ ]Skipped
Run test: [x]Passed [ ]Failed [ ]Skipped

Signed-off-by: Parichay Kapoor<EMAIL_ADDRESS>cc. @myungjoo

:memo: TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #1245. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://nnstreamer.mooo.com/.

Tensorflow2-lite header is dependent on flatbuffer, so flatbuffers-devel
is additionally added as a build-dependency.

If so, tensorflow2-lite-devel should declare Requires: flatbuffers-devel.
Applications (nntrainer is "application" of tensorflow2-lite-devel) should not be responsible of declaring secondary dependencies.

Tensorflow2-lite header is dependent on flatbuffer, so flatbuffers-devel
is additionally added as a build-dependency.

If so, tensorflow2-lite-devel should declare Requires: flatbuffers-devel.
Applications (nntrainer is "application" of tensorflow2-lite-devel) should not be responsible of declaring secondary dependencies.
If tensorflow2 packaging is not in our control, I'd accept declaring secondary dependencies, but we have control on tensorflow2 packaging of both Ubuntu and Tizen  .

I will update the tizen and ubuntu build of tensorflow2-lite, and wait for the updated package to be published.

When you've submitted changes, you'll need to tell integrators to merge and submit-build (SR in tizen.org, "Build Now" in launchpad.net) right away.

:octocat: cibot: @kparichay, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnstreamer.mooo.com/nntrainer/ci/repo-workers/pr-checker/1245-202106041131470.098043918609619-cf880b9e852b69b7f5f866b8f81e46647a268fd9/.

:octocat: cibot: @kparichay, A builder checker could not be completed because one of the checkers is not completed. In order to find out a reason, please go to http://nnstreamer.mooo.com/nntrainer/ci/repo-workers/pr-checker/1245-202106041215130.69966006278992-be12546bb5b313ae33d77b810c5e8543258e8825/.

Verified successful build for TIZEN x86_64 and armv7l, and ubuntu on local PC with updated tensorflow2-lite.


User refactoring

Start review on this code

Codecov Report

Merging #205 into develop will increase coverage by 0.74%.
The diff coverage is 76.72%.

@@              Coverage Diff              @@
##             develop     #205      +/-   ##
=============================================
+ Coverage      63.38%   64.13%   +0.74%
- Complexity       995     1048      +53
=============================================
  Files            104      109       +5
  Lines           4228     4472     +244
  Branches         243      260      +17
=============================================
+ Hits            2680     2868     +188
- Misses          1443     1493      +50
- Partials         105      111       +6

Impacted Files
Coverage Δ
Complexity Δ

...nico/socialsoftware/tutor/auth/AuthController.java
0.00% <0.00%> (ø)
0.00 <0.00> (ø)

...o/socialsoftware/tutor/auth/FenixEduInterface.java
0.00% <0.00%> (ø)
0.00 <0.00> (ø)

...co/socialsoftware/tutor/auth/JwtTokenProvider.java
39.13% <0.00%> (ø)
3.00 <0.00> (ø)

...ialsoftware/tutor/course/api/CourseController.java
0.00% <0.00%> (ø)
0.00 <0.00> (ø)

...co/socialsoftware/tutor/impexp/domain/Visitor.java
10.52% <0.00%> (-0.59%)
2.00 <0.00> (ø)

...nico/socialsoftware/tutor/user/dto/StudentDto.java
20.68% <20.00%> (+20.68%)
1.00 <0.00> (+1.00)

...tecnico/socialsoftware/tutor/user/dto/UserDto.java
31.42% <31.42%> (ø)
2.00 <2.00> (?)

...ico/socialsoftware/tutor/user/dto/AuthUserDto.java
44.92% <78.26%> (-19.66%)
10.00 <2.00> (ø)

...ico/socialsoftware/tutor/user/domain/AuthUser.java
80.32% <80.32%> (ø)
30.00 <30.00> (?)

...tecnico/socialsoftware/tutor/user/UserService.java
80.00% <82.14%> (-1.52%)
28.00 <5.00> (-5.00)

... and 25 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a17ba33...57cea8f. Read the comment docs.


UI CSS being overridden by page styling

Several styles are being overwritten by page stylesheets.
Fix: add the cssOrigin: "user" key when calling insertCSS in the background script.

Closed: ongoing issue.

Permanently closed since #22.


[BUG] System.Collections.Generic.KeyNotFoundException: The given key 'content-platform-dictionaries.zip' was not present in the dictionary.

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.
[X] I have thoroughly looked through the available Wiki articles and could not find a solution to my problem.
[X] I am using the latest version of Bloxstrap.
[ ] I did not answer truthfully to all the above checkboxes.

Bloxstrap Version
v2.8.1
What problem did you encounter?
i looked at the wiki btw to see if this error in specific was in the wiki but i guess it isnt

Bloxstrap Log
2024-10-31T02:40:11Z [App::OnStartup] Starting Bloxstrap v2.8.0
2024-10-31T02:40:11Z [App::OnStartup] Compiled Wednesday, 16 October 2024 at 5:44:37 AM from commit 42ffec26cf2e080fe22210305a08fcb286aeb577 (tag/v2.8.0)
2024-10-31T02:40:11Z [App::OnStartup] Loaded from %UserProfile%\AppData\Local\Bloxstrap\Bloxstrap.exe
2024-10-31T02:40:11Z [App::OnStartup] Temp path is %UserProfile%\AppData\Local\Temp\Bloxstrap
2024-10-31T02:40:11Z [App::OnStartup] WindowsStartMenu path is %UserProfile%\AppData\Roaming\Microsoft\Windows\Start Menu\Programs
2024-10-31T02:40:11Z [LaunchSettings] Identifier 'player' is active with data
2024-10-31T02:40:11Z [Logger::Initialize] Initializing at %UserProfile%\AppData\Local\Bloxstrap\Logs\Bloxstrap_20241031T024011Z.log
2024-10-31T02:40:11Z [Logger::Initialize] Finished initializing!
2024-10-31T02:40:11Z [JsonManager<Settings>::Load] Loading from %UserProfile%\AppData\Local\Bloxstrap\Settings.json...
2024-10-31T02:40:11Z [JsonManager<Settings>::Load] Loaded successfully!
2024-10-31T02:40:11Z [JsonManager<State>::Load] Loading from %UserProfile%\AppData\Local\Bloxstrap\State.json...
2024-10-31T02:40:11Z [JsonManager<State>::Load] Loaded successfully!
2024-10-31T02:40:11Z [FastFlagManager::Load] Loading from %UserProfile%\AppData\Local\Bloxstrap\Modifications\ClientSettings\ClientAppSettings.json...
2024-10-31T02:40:11Z [FastFlagManager::Load] Loaded successfully!
2024-10-31T02:40:11Z [LaunchHandler::LaunchRoblox] Initializing bootstrapper
2024-10-31T02:40:11Z [LaunchHandler::LaunchRoblox] Initializing bootstrapper dialog
2024-10-31T02:40:11Z [Bootstrapper::Run] Running bootstrapper
2024-10-31T02:40:11Z [Bootstrapper::SetStatus] Connecting to Roblox...
2024-10-31T02:40:11Z [Deployment::TestConnection<https://setup.rbxcdn.com>] Connecting...
2024-10-31T02:40:11Z [HttpClientLoggingHandler::ProcessRequest] GET https://setup.rbxcdn.com/versionStudio
2024-10-31T02:40:11Z [Deployment::InitializeConnectivity] Testing connectivity...
2024-10-31T02:40:11Z [HttpClientLoggingHandler::ProcessResponse] 200 OK https://setup.rbxcdn.com/versionStudio
2024-10-31T02:40:11Z [Deployment::InitializeConnectivity] Got https://setup.rbxcdn.com as the optimal base URL
2024-10-31T02:40:11Z [Bootstrapper::Run] Connectivity check finished
2024-10-31T02:40:12Z [Bootstrapper::CheckForUpdates] More than one Bloxstrap instance running, aborting update check
2024-10-31T02:40:12Z [Bootstrapper::GetLatestVersionInfo] Got channel as production
2024-10-31T02:40:12Z [Deployment::GetInfo] Getting deploy info for channel production
2024-10-31T02:40:12Z [HttpClientLoggingHandler::ProcessRequest] GET https://clientsettingscdn.roblox.com/v2/client-version/WindowsPlayer
2024-10-31T02:40:12Z [HttpClientLoggingHandler::ProcessResponse] 200 OK https://clientsettingscdn.roblox.com/v2/client-version/WindowsPlayer
2024-10-31T02:40:12Z [RegistryKeyEx::SetValueSafe] Writing '' to HKEY_CURRENT_USER\SOFTWARE\ROBLOX Corporation\Environments\RobloxPlayer\Channel\www.roblox.com
2024-10-31T02:40:12Z [HttpClientLoggingHandler::ProcessRequest] GET https://setup.rbxcdn.com/version-b7eebc919e96477a-rbxPkgManifest.txt
2024-10-31T02:40:12Z [HttpClientLoggingHandler::ProcessResponse] 200 OK https://setup.rbxcdn.com/version-b7eebc919e96477a-rbxPkgManifest.txt
2024-10-31T02:40:12Z [Bootstrapper::SetStatus] Upgrading {product}...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.RobloxApp.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Extracting RobloxApp.zip...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-avatar.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Extracting content-avatar.zip...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-configs.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Extracting content-configs.zip...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-fonts.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-models.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-sky.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Extracting content-fonts.zip...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-sounds.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Extracting content-models.zip...
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Finished extracting content-configs.zip
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Extracting content-sky.zip...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.shaders.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.ssl.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-textures2.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-textures3.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-terrain.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::DownloadPackage.content-platform-fonts.zip] Package is already downloaded, skipping...
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Finished extracting content-sky.zip
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Extracting content-sounds.zip...
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Finished extracting content-sounds.zip
2024-10-31T02:40:23Z [Bootstrapper::ExtractPackage] Extracting shaders.zip...
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Extracting ssl.zip...
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Finished extracting content-avatar.zip
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Extracting content-textures2.zip...
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Finished extracting ssl.zip
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Extracting content-textures3.zip...
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Finished extracting content-models.zip
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Extracting content-terrain.zip...
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Finished extracting content-terrain.zip
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Extracting content-platform-fonts.zip...
2024-10-31T02:40:24Z [Bootstrapper::DownloadPackage.content-platform-dictionaries.zip] Package is already downloaded, skipping...
2024-10-31T02:40:24Z [Bootstrapper::DownloadPackage.extracontent-places.zip] Package is already downloaded, skipping...
2024-10-31T02:40:24Z [Bootstrapper::DownloadPackage.extracontent-luapackages.zip] Package is already downloaded, skipping...
2024-10-31T02:40:24Z [Bootstrapper::DownloadPackage.extracontent-translations.zip] Package is already downloaded, skipping...
2024-10-31T02:40:24Z [Bootstrapper::DownloadPackage.extracontent-models.zip] Package is already downloaded, skipping...
2024-10-31T02:40:24Z [Bootstrapper::DownloadPackage.extracontent-textures.zip] Package is already downloaded, skipping...
2024-10-31T02:40:24Z [Bootstrapper::DownloadPackage.WebView2.zip] Package is already downloaded, skipping...
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Finished extracting shaders.zip
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Finished extracting content-platform-fonts.zip
2024-10-31T02:40:24Z [Bootstrapper::ExtractPackage] Extracting extracontent-places.zip...
2024-10-31T02:40:25Z [Bootstrapper::ExtractPackage] Finished extracting extracontent-places.zip
2024-10-31T02:40:25Z [Bootstrapper::ExtractPackage] Extracting extracontent-luapackages.zip...
2024-10-31T02:40:25Z [Bootstrapper::ExtractPackage] Finished extracting RobloxApp.zip
2024-10-31T02:40:25Z [Bootstrapper::ExtractPackage] Extracting extracontent-translations.zip...
2024-10-31T02:40:25Z [Bootstrapper::ExtractPackage] Finished extracting extracontent-translations.zip
2024-10-31T02:40:25Z [Bootstrapper::ExtractPackage] Extracting extracontent-models.zip...
2024-10-31T02:40:25Z [Bootstrapper::DownloadPackage.WebView2RuntimeInstaller.zip] Package is already downloaded, skipping...
2024-10-31T02:40:25Z [Bootstrapper::DownloadPackage.redist.zip] Package is already downloaded, skipping...
2024-10-31T02:40:25Z [Bootstrapper::SetStatus] Configuring {product}...
2024-10-31T02:40:25Z [Bootstrapper::ExtractPackage] Finished extracting content-textures3.zip
2024-10-31T02:40:25Z [Bootstrapper::ExtractPackage] Extracting extracontent-textures.zip...
2024-10-31T02:40:26Z [Bootstrapper::ExtractPackage] Extracting WebView2.zip...
2024-10-31T02:40:26Z [Bootstrapper::ExtractPackage] Finished extracting WebView2.zip
2024-10-31T02:40:26Z [Bootstrapper::ExtractPackage] Extracting redist.zip...
2024-10-31T02:40:28Z [Bootstrapper::ExtractPackage] Finished extracting extracontent-models.zip
2024-10-31T02:40:28Z [Bootstrapper::ExtractPackage] Finished extracting content-fonts.zip
2024-10-31T02:40:28Z [Bootstrapper::ExtractPackage] Finished extracting extracontent-luapackages.zip
2024-10-31T02:40:28Z [Bootstrapper::ExtractPackage] Finished extracting redist.zip
2024-10-31T02:40:40Z [Bootstrapper::ExtractPackage] Finished extracting extracontent-textures.zip
2024-10-31T02:41:20Z [Bootstrapper::ExtractPackage] Finished extracting content-textures2.zip
2024-10-31T02:41:20Z [LaunchHandler::LaunchRoblox] Bootstrapper task has finished
2024-10-31T02:41:20Z [LaunchHandler::LaunchRoblox] An exception occurred when running the bootstrapper
2024-10-31T02:41:20Z [App::FinalizeExceptionHandling] System.Collections.Generic.KeyNotFoundException: The given key 'content-platform-dictionaries.zip' was not present in the dictionary.
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at Bloxstrap.Bootstrapper.ExtractPackage(Package package, List`1 files)
   at Bloxstrap.Bootstrapper.<>c__DisplayClass33_1.<UpgradeRoblox>b__6()
   at System.Threading.Tasks.Task.InnerInvoke()
   at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)
   at System.Threading.ExecutionContext.RunFromThreadPoolDispatchLoop(Thread threadPoolThread, ExecutionContext executionContext, ContextCallback callback, Object state)
--- End of stack trace from previous location ---
   at System.Threading.Tasks.Task.ExecuteWithThreadLocal(Task& currentTaskSlot, Thread threadPoolThread)
--- End of stack trace from previous location ---
   at Bloxstrap.Bootstrapper.UpgradeRoblox()
   at Bloxstrap.Bootstrapper.Run()
   at Bloxstrap.Bootstrapper.Run()

just reinstall it worked for me

You're quite clearly not using 2.8.1 though lol


Multilevel test example

@annefou @jmdelouis, @keewis , I could save/plot 3 levels, I try with ww3 data now

got 2 levels of ww3 data plotted, but the example .npy data has problem.


Mark TestAnalysisWritesStatus as requiring custom setup

Since test setup has
		Setup(istio.Setup(nil, func(_ resource.Context, cfg *istio.Config) {
			cfg.ControlPlaneValues = `
values:
  pilot:
    env:
      PILOT_ENABLE_STATUS: true
  global:
    istiod:
      enableAnalysis: true
`
		})).

should be marked as requiring custom setup

@Monkeyanator: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name
Commit
Details
Required
Rerun command

release-notes_istio
78f5e975bb6e21412b27d5e2188713dd084ba0f8
link
true
/test release-notes_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/test release-notes_istio


ATX headings without following empty line get rendered wrong

Following Markdown:
### Composition
You may have noticed the line with the note: _Where to begin the query?_
In our application, we have _one_ normalized-state. So we want _one_ query to get _one_ tree to pass to our render function.

Coming back to our complete code from before, we can replace the `person` function with our component:

gets rendered like:

(Note: the ticks around person are part of my stylesheet, it is not a bug.)

This happens for the same reason as #14 happens, to quote a reply I made there:

This is by design. Clarktown parses blocks, and blocks are created by having two line breaks in-between. Thus, your example is just one block of text as far as Clarktown is concerned and not two different ones. And since the block starts with regular text, it determines it's just a paragraph and not a code block, which starts with with the uptick characters. You really want to have two line breaks between blocks.
To make this work would require some significant rewrite of the underlying architecture as the whole premise is built under the idea that blocks are separated by two newlines.

On further thought, I think I might be able to figure out a way with the a slight architecture change to make this work. I'll see if I can do it the coming weekend. Same goes for #14.


on search, all plasmids and strains appear double

When you search for a specific strain or plasmid, or a family with same stem of name, they are all listed double.
Search on plasmids e.g. for pDM314

oh no, it still pops up. First search all is fine, then clicked on a strain and publication. Then I searched for the same again and the strain I was on appears twice in the list
When you click on the box next to it then, both identical strains are clicked, so the database knows it's the identical strain. Strange behavior.

This seems to be only happening for plasmids for some reason. Will need to write more unit tests for the catalogs to diagnose.