Mapping error

Hi,
I installed BitMapperBS successfully and runed it without error. However, it seemed to have a conflict in the output  *.sam.
data resource：PE100
Mapping information of the output *.sam:

Header information of the output *.sam:

problems: the Mapping position was 89997，reads length was 100bp, 89997+100=90097, this exceeded the length of chr9_gl000198_random, which was 90085. That resulted in a mistake when remove duplicates:

Hi,
I installed BitMapperBS successfully and runed it without error. However, it seemed to have a conflict in the output *.sam.
data resource：PE100
Mapping information of the output *.sam:

Header information of the output *.sam:

problems: the Mapping position was 89997，reads length was 100bp, 89997+100=90097, this exceeded the length of chr9_gl000198_random, which was 90085. That resulted in a mistake when remove duplicates:

This is a bug, I will fix it immediately. Thanks for your help.
If you do not want to run BitMapperBS again, maybe you can just delete these two lines (read1 and read2 of CL100089207L1C010R068_396600/2) in output SAM file.
PS: BitMapperBS (version <IP_ADDRESS>) now can support output result in BAM format directly. It will save a lot of time.

Hi,
I installed BitMapperBS successfully and runed it without error. However, it seemed to have a conflict in the output *.sam.
data resource：PE100
Mapping information of the output *.sam:

Header information of the output *.sam:

problems: the Mapping position was 89997，reads length was 100bp, 89997+100=90097, this exceeded the length of chr9_gl000198_random, which was 90085. That resulted in a mistake when remove duplicates:

I have tested Picard successfully with ALIDATION_STRINGENCY=LENIENT or SILENT a moment ago. If you do not want to run BitMapperBS again, I recommend you to run Picard like this. Actually, it is It's extremely rare when a read hangs off of one chromosome and on to another.

I tried the method that you suggested before and it did't remove duplicates actually.
this was the code:

results:

I used old version of BitMapperBS (version <IP_ADDRESS>). The output file was a sorted BAM file after sorted by samtools. But it still did't work.
this was the code:

results:

In my experiments, Picard will "output Marking 23291334 records as duplicates". Have you ever seen this?
I have tested your command again in my datasets, and I could not reproduce this problem. Could you please re-align your dataset with current version of BitMapperBS (version <IP_ADDRESS>)? I note that your dataset only includes about 10 million reads. The new version BitMapperBS only needs a few minutes to process them, and can directly output BAM file (using "--bam" and "-o output.bam" options).
If this problem still exist, could you please send me a small part of your dataset? It would be a great help for me.
Thanks very much!

In my experiments, Picard reported "Marking 0 records as duplicates"

And I will try to install BitMapperBS (version <IP_ADDRESS>).
Thank you

There was only BitMapperBS (version <IP_ADDRESS>) in this web.

There was only BitMapperBS (version <IP_ADDRESS>) in this web.

Have you download source code of BitMapperBS? Make it from source code will generate BitMapperBS (<IP_ADDRESS>).

I had downloaded the source code of BitMapperBS, but didn't installed it successfully.

I had downloaded the source code of BitMapperBS, but didn't installed it successfully.

I am surprised by this error. Could you please compile "BitMapperBS_v<IP_ADDRESS>.zip" first?
In addition, cloud you please send me the information of your CPU and Linux system? This problem may be caused by old CPU without AVX2.

I had tried to compile "BitMapperBS_v<IP_ADDRESS>.zip" and could not install BitMapperBS_v<IP_ADDRESS>.

I had tried to compile "BitMapperBS_v<IP_ADDRESS>.zip" and could not install BitMapperBS_v<IP_ADDRESS>.

Is this the error information reported by BitMapperBS?

I had tried to compile "BitMapperBS_v<IP_ADDRESS>.zip" and could not install BitMapperBS_v<IP_ADDRESS>.

I know what is mean ...
which gcc and linux are you using?

I had tried to compile "BitMapperBS_v<IP_ADDRESS>.zip" and could not install BitMapperBS_v<IP_ADDRESS>.

Could please send me the error information reported when build BitMapperBS_v<IP_ADDRESS>?

OK，I will sen you tomorrow
Thank you for your reply.
获取 Outlook for iOShttps://aka.ms/o0ukef

收件人: chhylp123<EMAIL_ADDRESS>发送时间: 星期一, 十二月 10, 2018 6:08 下午
收件人: chhylp123/BitMapperBS
抄送: vanessayang0927; Author
主题: Re: [chhylp123/BitMapperBS] Mapping error (#5)
I had tried to compile "BitMapperBS_v<IP_ADDRESS>.zip" and could not install BitMapperBS_v<IP_ADDRESS>.
[image]https://user-images.githubusercontent.com/45344513/49724894-6939d000-fca5-11e8-936b-38765ff04eee.png
Could please send me the error information reported when build BitMapperBS_v<IP_ADDRESS>?
―
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/chhylp123/BitMapperBS/issues/5#issuecomment-445760053, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ArPnAQRVMZu6cnqjZ4uuMsUiYcRfHbN_ks5u3jKKgaJpZM4ZKfrk.

OK，I will sen you tomorrow Thank you for your reply. 获取 Outlook for iOShttps://aka.ms/o0ukef
…
________________________________ 收件人: chhylp123<EMAIL_ADDRESS>发送时间: 星期一, 十二月 10, 2018 6:08 下午 收件人: chhylp123/BitMapperBS 抄送: vanessayang0927; Author 主题: Re: [chhylp123/BitMapperBS] Mapping error (#5) I had tried to compile "BitMapperBS_v<IP_ADDRESS>.zip" and could not install BitMapperBS_v<IP_ADDRESS>. [image]https://user-images.githubusercontent.com/45344513/49724894-6939d000-fca5-11e8-936b-38765ff04eee.png Could please send me the error information reported when build BitMapperBS_v<IP_ADDRESS>? ― You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#5 (comment)>, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ArPnAQRVMZu6cnqjZ4uuMsUiYcRfHbN_ks5u3jKKgaJpZM4ZKfrk.

Thanks very much.
If possible, could you please enter the directory of BitMapperBS (<IP_ADDRESS>), and make it by:
make AVX2=0
This will generate BitMapperBS (<IP_ADDRESS>) from source code without AVX2.
Thanks again for your help to debug BitMapperBS. If you like, I will kindly acknowledge you in my paper.

Have you install zlib, libbz2 and liblzma libraries？Could you please install these libraies first, and then try: make AVX2=0.
I have searched this problem on internet, someone said this is the bug of gcc 4.8. So if you still want to make BitMapperBS with AVX2, could please upgrade it? gcc 4.8 is very old now.

Hi,
I installed BitMapperBS_v<IP_ADDRESS> successfully and tried to remove duplicates in output *bam by Picard. Picard still reported "Marking 0 records as duplicates" to stdout, but the duplicates existed actually.
duplicates existed in sorted output bam file:

my code:

results:

I will try to upload test.bam to github. PS: I had used Picard to remove dup in BSmap output bam file without error.

Hi,
I installed BitMapperBS_v<IP_ADDRESS> successfully and tried to remove duplicates in output *bam by Picard. Picard still reported "Marking 0 records as duplicates" to stdout, but the duplicates existed actually.
duplicates existed in sorted output bam file:

my code:

results:

I will try to upload test.bam to github. PS: I had used Picard to remove dup in BSmap output bam file without error.

From the alignment results of BitMapperBS, it seems that there are no problems ...
I will wait your test.bam. Thank you very much.
By the way, how do you solve the problems when built BitMapperBS from source code? I hope I can write a tip in README.

this is the website of test.bam: https://github.com/vanessayang0927/BitMapperBS/blob/master/test.bam

#install BitMapperBS-<IP_ADDRESS>
wget https://github.com/chhylp123/BitMapperBS/archive/master.zip
unzip master.zip
cd BitMapperBS-maste
export PATH=/path/to/CMake/v3.13.1/bin:$PATH
make -j 8
cd ..
mkdir v<IP_ADDRESS>
cp -a BitMapperBS-master/bitmapperBS v<IP_ADDRESS>
cp -a BitMapperBS-master/psascan v<IP_ADDRESS>
cp -a BitMapperBS-master/htslib_aim v<IP_ADDRESS>

BitMapperBS was made successfully with a administrator access

BitMapperBS was made successfully with a administrator access

Well, I think I know what happen ... It is a minor problem. I will solve it this evening.

this is the website of test.bam: https://github.com/vanessayang0927/BitMapperBS/blob/master/test.bam

Thanks for your data. This is a minor problem in BitMapperBS. I have already fixed it in new version of BitMapperBS (<IP_ADDRESS>). In addition, the reads which are mapped off the end of the reference are reomved from the alignment result in BitMapperBS (<IP_ADDRESS>). Thus, you can run Picard.MarkDuplicates normally without ALIDATION_STRINGENCY=LENIENT or SILENT. I highly recommend  you to try the new version of BitMapperBS (<IP_ADDRESS>) from github.
When align paired-end reads, this problem is caused by inconsistent QNAMEs (Read ID) of read 1 and read 2 in BAM file. For example, given a paired-end read in your dataset, the Read IDs of read1 and read2 are  "CL100089207L1C015R033_267765/1" and "CL100089207L1C015R033_267765/2", respectively. In BAM file generated by previous BitMapperBS, QNAMEs of read1 and read2 are still "CL100089207L1C015R033_267765/1" and "CL100089207L1C015R033_267765/2", respectively. Since the QNAMEs of read1 and read2 are different, Picard.MarkDuplicates does not think read1 and read2 are a paired-end read. In this case, Picard.MarkDuplicates cannot identify any deuplicate paired-end read. In new version of BitMapperBS (version <IP_ADDRESS>), the QNAMEs of both read1 and read2 are "CL100089207L1C015R033_267765". Therefore, Picard.MarkDuplicates thinks they are a paired-end read.

Hi,
Hi,
I installed BitMapperBS (<IP_ADDRESS>) successfully and remove duplicates without error. But there was a contradiction between the output log file and output bam file:
the number of unique mapped Reads in log file

the number of unique mapped Reads in bam file

Hi,
I installed BitMapperBS (<IP_ADDRESS>) successfully and remove duplicates without error. But there was a contradiction between the output log file and output bam file:
the number of unique mapped Reads in log file

the number of unique mapped Reads in bam file

Hi,
I installed BitMapperBS (<IP_ADDRESS>) successfully and remove duplicates without error. But there was a contradiction between the output log file and output bam file:
the number of unique mapped Reads in log file

the number of unique mapped Reads in bam file

It is a strange problem. Colud you please check the number of lines in the two original fastq files you had? Perhaps one has a few extra reads.
Or could you please upload a sample dataset?
Thanks very much.

Thanks for your reply!
Considering that error did not appear in v<IP_ADDRESS>, I did not think that error because of the sequencing plat.

Thanks for your reply!
Considering that error did not appear in v<IP_ADDRESS>, I did not think that error because of the sequencing plat.

Do you have Wechat? Maybe we can discuss by Wechat. I really want know what happen.

Thanks for your reply!
Considering that error did not appear in v<IP_ADDRESS>, I did not think that error because of the sequencing plat.

Do you have Wechat? Maybe we can discuss by Wechat. I really want know what happen.

Wechat Account：yangqin0927123

Wechat Account：yangqin0927123

I cannot find yangqin0927123 in Wechat... Is your Wechat ID right?


darth vader / rey / capt marvel watches need some UI work

These are touch screen but seem to be different than the venu or vivoactive in the way you interact with it. Also they share the same screen size as devices with buttons but don't really have any so the layouts need to be tweaked a little.

I'll adjust the superhero watch UIs.

Top-right button - Settings
Bottom-right button - Back
Click on screen - toggle / enter Steak Menu
Scroll down - Smoking menu
Scroll up - Settings menu / Smoke menu
DurationPicker - This is working OK
Click on arrows - increase/decrease
Top-right button - go to the next time selection / finalize selection
Top-bottom button - go to previous time selection
I'll fix this tomorrow.

Vivoactive4S uses the same type of interface. Fixed.

Vivoactive4 as well. Fixed.

First Avenger as well. Fixed.

Venu as well. Fixed

I'm sure more of these type of watches will come out as we test the rest. Will keep this one open.

Closing this. All the watches with weird button layouts are fixed individually.


Using RxUI with Rx v3.x possible?

Hi,
I have a multi-project solution which uses Rx v3.1.1. In a new WPF application I would like to use RxUI, but RxUI v7.1.0 forces me to install Rx v2.2.5. Due to compatibility reasons I'm not allowed to use Rx v2.2.5 anymore.

Can RxUI v7.1.0 be used with Rx v3.1.1?
Or do I have to drop RxUI for now and wait for RxUI v8?
When will RxUI v8 be available?

Thanks!

Howdy @dotNZ. I've put #1190 #1123 on hold until v7 stabilises as migration to netstandard at this stage will create a rift of those who can migrate and those who cannot.  You are feeling this right now (you belong in the "can migrate"). I specifically do not want to be concurrently supporting v7 (rx2) and v8 (netstandard/rx3) at the same time.
Check out the netstandard branch on GitHub if you would like to help with progressing v8; let's talk.

@ShannonKasperNoesis, @oliverw, @Mephistofeles and @dotNZ. I've put #1190 on hold until v7 stabilises as migration to netstandard at this stage will create a rift of those who can migrate and those who cannot.  You are feeling this right now (you belong in the "can migrate"). I specifically do not want to be concurrently supporting v7 (rx2) and v8 (netstandard/rx3) at the same time as I don't have the time to be able to do so.  Check out the netstandard branch on GitHub if you would like to help with progressing v8; let's talk.  We would <3 to and intend to grow our team in 2017.

Closing this one out. If you need Rx3.11 please start migrating towards the pre-release that's available on NuGet.org


build: merge stage to master

:tada: This PR is included in version 0.16.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Question: Adding Secrets to Already Encrypted Secrets File

I have a use case where it would be ideal if there was some way to add new secrets to an existing secrets file that is encrypted. I know this is possible by decrypting the file then adding and then re-encrypting, but I am using Azure Keyvault and many folks who may need to have access to add new secret values will not have access to use the key in the vault to decrypt, so they cannot decrypt and re-encrypt all of the secrets.
Is this somehow possible today?

@cdunford I think the only possible way would be to split the secrets file into two, one with the key that only admins have access to, and another with the key everyone has access to.
Not sure if this works with helm-secrets, but I remember that sops - the tool used by helm-secrets under the hood to enc/dec secrets - had the ability to use a dedicated key for each encrypted secrets file. So hopefully it just work.
Another way would be to enhance vals. It's the secrets manager embedded in Helmfile to turn any yaml values formatted like ref+<provider name>://id/of/the/secret/you/want/to/load into actual secrets values.
This way, you don't need to re-encrypt the secrets file to add/remove a single secret.
It doesn't support Azure KeyVault or a kind of secrets manager in Azure yet https://github.com/variantdev/vals#suported-backends so hopefully you could submit a pull request OR help me to implement/test the Azure provider for vals to support your usecase.

I am using the multiple secrets files approach currently and it works fine. My use case doesn't actually involve different keys. It's the same key for all secrets, it's just no one has access to use the key in the vault to decrypt, they can just use it to encrypt.
I wasn't aware of the other means to load secrets directly from the values. Is that documented somewhere (maybe I just missed it)?

@cdunford It's available since https://github.com/roboll/helmfile/pull/906 but it isn't documented yet.

it's just no one has access to use the key in the vault to decrypt, they can just use it to encrypt.

Well then, I guess the only way would be to never decrypt/re-encrypt it. In other words, create a new secret file for each newly added secret value.


Phrokka gff contig name

phrokka version: 0.1.0
Python version: 3.9
Operating System: MacOS

Description
Small thing, but I noticed that in the .gff output that the names for the CDS and the tRNA hits are different - seems trimmed.

Should be fixed with v0.1.7 removal of hhsuite


return hyper-esque errors

update cloud.hyper adapters to return hyper-esque errors on managed errors and rejected promises on an unhandled exception.
Each adapter method in the case of handled exceptions should return a resolved promise that resolves to a hyper error
A hyper error consists of:
ok: false
msg: optional: string
status: optional: integer
Unhandled exceptions the adapter should return a rejected promise. We should get this oob.

dupe of #28


Cleanup the rdfs:domain and rdfs:range relation.

When two types A and B are declared as the domain or range of a property
such that A is a supertype of B, then only A should be kept in the
relation (since it is subtype-transitive).

Actually, for the range we want to keep the smallest subclass(es).


App exits on PostgreSQL Geometry field select

System information:

Ubuntu 20.04 LTS
DBeaver Version <IP_ADDRESS>007192031

Connection specification:

PostgreSQL 12.2 (Ubuntu 12.2-4) on x86_64-pc-linux-gnu, compiled by gcc (Ubuntu 9.3.0-8ubuntu1) 9.3.0, 64-bit
PostgreSQL JDBC Driver
Do you use tunnels or proxies (SSH, SOCKS, etc)? - No

Describe the problem you're observing:
Application exists in 1 sec after I click on a geometry type field value cell.
Steps to reproduce, if exist:

Open DB Table through Navigator -> Right Click -> View Data. Table needs to have a geometry field AND in my case PostGIS extension is enabled.
Click on that geometry field value if you already have a record OR click Add New Row button and then click on that field.

Include any warning/errors/backtraces from the logs
eclipse.buildId=unknown
java.version=1.8.0_252
java.vendor=Private Build
BootLoader constants: OS=linux, ARCH=x86_64, WS=gtk, NL=en_US
Command-line arguments:  -os linux -ws gtk -arch x86_64
org.eclipse.core.resources
Warning
Sun Jul 26 17:09:00 EEST 2020
The workspace exited with unsaved changes in the previous session; refreshing workspace to recover changes.

Can you give me the DDL & some sample data?
I try to this in my environment, DBeaver was not exited and processed well.

Can you give me the DDL & some sample data?
I tried to this in my environment, DBeaver was not exited and processed well.

Sorry, it looks like I've made a mistake - the field is "geography" type but not "geomerty"
CREATE TABLE public.tasks (
id bigserial NOT NULL,
user_id int8 NOT NULL,
name varchar(150) NOT NULL,
description text NULL,
geo_point geography NULL,
is_active bool NOT NULL DEFAULT true,
status_id int8 NOT NULL,
created_at timestamp NULL,
updated_at timestamp NULL,
CONSTRAINT routes_pkey PRIMARY KEY (id)
);

@thewwwizard Can you give me a query example for some records.

@hsseo0501 I'm able to reproduce it even with no data, when I click on add new row and then select geography field to add value.
Here is an example record:
INSERT INTO public.tasks (user_id,"name",description,geo_point,is_active,status_id,created_at,updated_at) VALUES
(1,'Task 1','Description 1','POINT (-71.060316 48.432044)',true,1,NOW(),NOW());

Most probably the issue happens due to a browser integration. Could you please find and send log file: hs_err_pidXXXX.log  ?

Most probably the issue happens due to a browser integration. Could you please find and send log file: hs_err_pidXXXX.log ?

Hi @kseniiaguzeeva . Wondering how quickly you reply guys! Bravo!
Simulated that error again and attaching last log file here.
hs_err_pid624074.log
..

not reproducible in 21.3.1


Investigate impact of reboot and cloud init interaction

I think the cloud init script might re run on reboot.  Not sure what that would do.  It'd be good to understand the impact here.  We're going to see more of these as we do more operational work.

Running on reboot (in addition to first boot) is NOT the default behaviour, however this can be changed if required.
[https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/user-data.html#user-data-shell-scripts]

We would not want it to run.  If it doesn’t I think we’re all good.  I just had this feeling that it might but I can’t remember where I last observed that.

Enabled systemctl neo4j since on reboot neo4j was not coming up
https://github.com/neo4j-partners/amazon-cloud-formation-neo4j/commit/d2d34daa8bdcb1e324e88583e0403e0337e11e51


[WIP] Implement event activity API

@pengwynn Work in progress, but have a look to see if its going the way you want.

@dhruvsinghal Looking good. I think if you cherry-pick 419bb92daed29a581361a8c1f0e3072273d3c1ab or rebase over master, this will go :green_apple:.

I would love to play with the Event API, any ETA for merging this PR ?


pull request for e2e test at 02/05/2017 04:56:09

:white_check_mark: Validation status: passed
For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Read Flow consumed without a consumer

Hello dev, this is not really an issue and maybe I didn’t understand correctly the nature of the channelFlow used in the readByteArrayStream function. In few words I created a Service to share the socket between activities but switching from an activity to another I loose some message. I tried to write on stackoverflow but it seems that no one knows your lib. Take a look here to check the code I wrote: https://stackoverflow.com/questions/66965201/i-loose-data-from-bluetooth-moving-between-activities-and-using-services?r=SearchResults
Anyway a read example would be appreciated.
Thank you

Hello, sorry about the incomplete docs. I made this lib to support one of my projects. Never thought there would be interest from other developers.
Please give me some time to look into your issue and try to post back. Keep in mind that many Bluetooth devices need some sort of queueing + delay mechanism that this lib does not support.


[FR]: Support multiple nested snapshot directories

What is the current behavior?
Right now, the write_source_file underpinning the snapshot update command will take the first __snapshot__ it finds and write that back. It will ignore any nested ones, requiring the end user to have multiple build files and other glue logic to make it work. On a small project, this is a lot of overhead.
Describe the feature
❯ tree web/babel-preset
web/babel-preset
├── BUILD.bazel
├── README.md
├── __snapshots__
│   └── index.test.js.snap
├── deps.bzl
├── index.js
├── index.test.js
├── lazy-importer
│   ├── README.md
│   └── src
│       ├── __snapshots__
│       │   └── index.test.js.snap
│       ├── index.js
│       └── index.test.js
├── node_modules
│   └── @babel
│       ├── core -> ../.pnpm/@babel+core@7.18.6/node_modules/@babel/core
│       └── template -> ../.pnpm/@babel+template@7.18.6/node_modules/@babel/template
├── package.json
├── pnpm-lock.yaml
├── registerBabel.js
└── yarn.lock

I should be able to have one jest_test rule at the root BUILD.bazel file in this tree and it should write back to every folder that has a __snapshot__ folder until such time I split out something and have another build file nested.
Fund our work

[ ] Sponsor our open source work by donating a feature bounty

Discussion for context https://bazelbuild.slack.com/archives/CEZUUKQ6P/p1674589681819949

This is now supported.


Quantum Chess: Check reservation status

One feature that might be cool is to be able to play on a real quantum computer during specific times in the day and execute moves on hardware opportunistically.
Using the API (https://quantumai.google/cirq/google/engine) and reservation functionality similar to this colab (https://github.com/quantumlib/Cirq/blob/master/docs/tutorials/google/reservations.ipynb), we could add in some functions to check whether the processors are available.  The function could take in a ordered list of processor ids and return which processor(s) are available to run on.
For bonus points, it would be good to use timing information from quantumlib/ReCirq#130 to keep track of how long it takes to run circuits on each processor as a proxy for how busy the processor is and use that to determine the best processor.

Closing this, as we don't have a reservation system right now.


OFA-Large vs. refcoco_large_best.pt in evaluation

How the Pretraining model OFA-Large could be used for inference/evaluation over one of the tasks such as visual grounding?
The evaluation scripts addressing the fine-tuned model s.a. refcoco_large_best.pt and not the OFA-Large, and not supportred in the code when executing evaluate_refcoco.sh. In the code
It can't infer the relevant task out of the ofa_large.pt model in :
models, saved_cfg, task = checkpoint_utils.load_model_ensemble_and_task(
    utils.split_paths(cfg.common_eval.path),
    arg_overrides=overrides,
    suffix=cfg.checkpoint.checkpoint_suffix,
    strict=(cfg.checkpoint.checkpoint_shard_count == 1),
    num_shards=cfg.checkpoint.checkpoint_shard_count,

How is it possible to reproduce the ACC50% results reported in table #1 for refCOCO for instance, over the pre-trained model?
Thanks.

@hanochk Hi, the results of Table 1 are based on fine-tuned models. We have updated the code to support evaluation for the pretrained models, please check out the latest code and have a try. (see run_scripts/refcoco/evaluate_refcoco_zeroshot.sh)


Correct translator

Come again. Your translator implementation gives different results with different batch size. It is due to that you repeat the Variable in wrong way.  And I correct this error.

Hi @ZiJianZhao,
I am so sorry that I miss this pull request and commit with my own ...
Thank you so much for the correction and insight!
Yu-Hsiang


[FEATURE] Create an ingest pipeline for neural search

Is your feature request related to a problem?
Implement workflow set up interface to facilitate the creation of an ingest pipeline for semantic search. The implementation should construct an ingestion pipeline using the text_embedding pre-processor which handles transforming IngestDocuments into vector embeddings prior to being indexed into a KNN index and stores the pipeline ID within the GlobalContext index. This pipeline ID will then be used to set as the default_pipeline of the KNN index.
The implementation should parse the use case template to retrieve the following inputs for the text_embedding pre-processor:

input_field_name
output_field_name

The implementation should leverage the provided client to read the model_id from the global context.
Using this information, create the PutPipelineRequest payload:
PUT _ingest/pipeline/neural-pipeline
{
  "description": "An example neural search pipeline",
  "processors" : [
    {
      "text_embedding": {
        "model_id": "<model_id>",
        "field_map": {
           "<input_field_name>": "<output_field_name>"
        }
      }
    }
  ]
}

Do you have any additional context?
Creation of the ingest pipeline should be done via a PutPipelineRequest using the transport client. Reference : https://github.com/opensearch-project/geospatial/blob/main/src/main/java/org/opensearch/geospatial/action/upload/geojson/PipelineManager.java#L92

@joshpalis this issue shouldn't just talk about neural pipeline but any ingest pipeline which we can create. There should be a type associated while creating the pipeline. If it's neural and the processor is  text_embedding the above pipeline would be created then.


Add missing givenName and familyName for MA legislators

This PR adds the missing givenName and familyName fields for 36 legislators.  This will enable ProgressiveMass's Legislator Scorecard to stop using heuristics-based parsing of full names into first and last names, which is imperfect and risks culturally insensitive errors. https://github.com/ProgressiveMass/legislator-scorecard/issues/90

@NewAgeAirbender  is this good to merge?

Yup! Looks good, thanks for doing this!


recording: missing startTime for network logs

:bulb: Motivation and Context
https://posthog.slack.com/archives/C03PB072FMJ/p1726143908027679
https://github.com/PostHog/posthog/blob/8b8fe95031463e68c47dd6d70f3ec5a23537c498/frontend/src/scenes/session-recordings/apm/playerInspector/ItemPerformanceEvent.tsx#L226
the label depends on start time or fetch time but we don't add it
:green_heart: How did you test it?
:pencil: Checklist

[X] I reviewed the submitted code.
[ ] I added tests to verify the changes.
[ ] I updated the docs if needed.
[X] No breaking change or entry added to the changelog.

closed in favor of https://github.com/PostHog/posthog-android/pull/175 since startTime is something else for the browser


Call your Params what they are

Similar to the the issue that I opened up about Blueprints (#12) let's call your params objects what they are instead of calling them structs. It will make it clearer to new users what's going on here.
I'd even extend it to Parameters

Do you mind if I change this to Input? While they're most often used as parameters, they're really a generalized "input" type that can parse basically anything, so I think that term is more correct. I'd maybe even extend this to call it them InputObject, since they're designed to parse an object (you can have inputs that are just a single item, for example, which are not sublcasses of SoberSwag::Struct yet)

Totally cool that. I like Input or InputObject. I think the latter's good if you're using ObjectSerializer for the outputs.


Remove note about mcrypt extension

The project hasn't used the mcrypt cipher since #205, so the note about it being needed may confuse users.

Thanks!

Thank you @mattstauffer


Hello! 👋 consider loco.rs

Hi!
This is a really cool project I'm using for myself. I'm also the author of Loco (https://loco.rs), the "Rails for Rust". You've got a lot of infrastructure (auth, db handling, etc.) that we've already built, and by chance we're also strongly relying on SeaORM
Give it a try, you can build a full app in 3-4 minutes: https://loco.rs/docs/getting-started/tour/
Maybe consider Loco for sero and save a lot of infra work

@jondot Hi!  I think i'll give it a try in another repository! Thank you for reaching me!


sqlite: fixing cve patch name

Motivation for this change
To make vulnix and similar tooling recognize the patch
correctly. This is only needed for 19.09 and only to make sure that vulnix does not raise
this issue again.
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nix-review --run "nix-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

Notify maintainers
cc @andir

This needs to target staging-19.09 branch


Fix error in getting-started code snippets

This fixes an error in the getting-started code where the order of elements was wrong:
correct order -> context.fragments.fragmentName.functions.functionName
"todos" is the name of the fragment on which the function should be called

You`re completely right, the order is wrong there.
Thank you for you contribution


 It still works from the command line, VS Code Stable build, and Atom Beta.

Reproduces without extensions: Yes
This was after the original update

This was after an additional update

This was after the update from 11/17/2017. I also tried turning off the extensions to see if that made a difference.

@joaomoreno I've left the setup as it is shown above. If you would like anymore info or for me to try anything, please let me know.
Thanks 👍

The latest version update gives this hint

You need to either install git from https://git-scm.com/ or to configure git.path to point to the location where you have installed it, if different from the default.

@mianfengchen I was able to get it to work after changing the git.path to point to the location of git.exe. It just seemed strange as the whole /git/ directory is set up as a path on my computer. And both the Stable and Insiders build work fine without setting up git.path in the settings.
And prior to the specific update the Git integration was working perfectly on the Insiders build without git.path set up. 🤔

We had to disable using git from the PATH: https://github.com/Microsoft/vscode/issues/32739#issuecomment-341177471


chore(autocomplete): remove component from docs and headless

What is it?

[ ] Feature / enhancement
[ ] Bug
[ ] Docs / tests

Description
@thejackshelton We can remove all this right?
Checklist:

[ ] My code follows the developer guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have made corresponding changes to the documentation
[ ] Added new tests to cover the fix / functionality

Looks good! Merging.


"Search your location" Section on the Homepage Design Improvement

The homepage design is nice but it can be improved by adjusting the design of "Search your location" area. By doing work on the styling and alignment of this area, I can make homepage look even better.
Screenshots
Current Design

Expected Updated Design

do not make changes on it


Auto-populate for PDAs using Methods namespace doesn't work for defined types

If a seed is a defined type the Auto-populate PDA's feature is failing which makes sense due to this.
I tried to give this a shot but couldn't make it work reliably. Decided to submit the issue in case somebody else is working on the same issue.

Fixed in https://github.com/coral-xyz/anchor/pull/2824.


time-picker css is empty

Reproduction link
https://stackblitz.com/edit/ng-zorro-antd-start-qk71ke
Steps to reproduce

Use time-picker
Import only for the component necessary styles

What is expected?
Time-picker should be displayed with the right style
What is actually happening?
Time-picker is displayed wrong

Environment
Info

ng-zorro-antd
9.1.0

Browser
Mozilla Firefox 75.0

time-picker/style/index.css is empty
It works if ng-zorro-antd.min.css is imported, but I only need the time-picker component

Currently, ng-zorro-antd only exports less file for the individual component, but you can't just import time-picker less because a component may rely on another component less. If you need time-picker to work, you should at least import:
@import "~ng-zorro-antd/style/entry.less";
@import "~ng-zorro-antd/date-picker/style/index.less";


Correct way of defining the voxel size for custom dataset

Hi,
I am trying to train a custom dataset using PointPillar. I have created the data according to what is shown on the custom dataset page. However, on training, I am getting the following error:
Exception has occurred: RuntimeError The size of tensor a (995400) must match the size of tensor b (1670400) at non-singleton dimension 1   File "/home/aagr656/OpenPCDet/pcdet/utils/loss_utils.py", line 60, in forward     pt = target * (1.0 - pred_sigmoid) + (1.0 - target) * pred_sigmoid   File "/home/aagr656/OpenPCDet/pcdet/models/dense_heads/anchor_head_template.py", line 128, in get_cls_layer_loss     cls_loss_src = self.cls_loss_func(cls_preds, one_hot_targets, weights=cls_weights)  # [N, M]   File "/home/aagr656/OpenPCDet/pcdet/models/dense_heads/anchor_head_template.py", line 217, in get_loss     cls_loss, tb_dict = self.get_cls_layer_loss()   File "/home/aagr656/OpenPCDet/pcdet/models/detectors/pointpillar.py", line 27, in get_training_loss     loss_rpn, tb_dict = self.dense_head.get_loss()   File "/home/aagr656/OpenPCDet/pcdet/models/detectors/pointpillar.py", line 14, in forward     loss, tb_dict, disp_dict = self.get_training_loss()   File "/home/aagr656/OpenPCDet/pcdet/models/__init__.py", line 44, in model_func     ret_dict, tb_dict, disp_dict = model(batch_dict)   File "/home/aagr656/OpenPCDet/tools/train_utils/train_utils.py", line 56, in train_one_epoch     loss, tb_dict, disp_dict = model_func(model, batch)   File "/home/aagr656/OpenPCDet/tools/train_utils/train_utils.py", line 180, in train_model     accumulated_iter = train_one_epoch(   File "/home/aagr656/OpenPCDet/tools/train.py", line 176, in main     train_model(   File "/home/aagr656/OpenPCDet/tools/train.py", line 231, in <module>     main() RuntimeError: The size of tensor a (995400) must match the size of tensor b (1670400) at non-singleton dimension 1
I have read all the existing issues related to it and one thing I came to know is that the issue can be solved by adjusting the voxel size and point cloud range. The point cloud range, in my understanding, depends upon the input data, so we can't really do much with it. So what remains is adjusting the voxel size.
Here are the details in the config file I am using:
`DATA_CONFIG:
BASE_CONFIG: tools/cfgs/dataset_configs/custom_dataset.yaml
POINT_CLOUD_RANGE: [-172.8, -172.8, -2, 172.8, 172.8, 38]
DATA_PROCESSOR:
- NAME: mask_points_and_boxes_outside_range
REMOVE_OUTSIDE_BOXES: True
    - NAME: shuffle_points
      SHUFFLE_ENABLED: {
        'train': True,
        'test': False
      }

    - NAME: transform_points_to_voxels
      VOXEL_SIZE: [1.44, 1.44, 40]
      MAX_POINTS_PER_VOXEL: 960
      MAX_NUMBER_OF_VOXELS: {
        'train': 32000,
        'test': 40000
      }`

The voxel size satisfies both the requirements I could identify from the existing issues. The point cloud range along z axis is 40 and point cloud range along X-Y axis/voxel size is a multiple of 16. Still, I am still getting the error. I have tried several other values of the point cloud range and voxel size satisfying the two conditions, but the error still persists.
I want to know what is the correct way of defining the voxel size? And, is there anything else I can check to get rid of this error?
Thanks,
Ajay

There are several reasons for this error, check the structure of the model used and make sure it gives an output size compatible with your labels size.

Law: point cloud range along x,y -axis / voxel_size is the multiple of 16.
range_x = x_max - x_min ; range_x / voxel_size mod 16 = 0
Notice that the second rule also suit pillar based detectors such as PointPillar and CenterPoint-Pillar.
x = [0, 70.4] ; y = [-40, 40]
x-axis: 70.4 - 0 = 70.4 / 0.1 = 704 mod 16 = 0
y-axis: 40 - (-40) = 80 / 0.1 = 800 mod 16 = 0
VOXEL [length, width, height] 0.05m = 5cm
VOXEL_SIZE: [0.05, 0.05, 0.1] # in meters


Missing configuration for WSS based Websocket

Description:
<parameter name="wss.ssl.protocols">TLSv1.1,TLSv1.2</parameter>
The above parameter of "wss.ssl.protocols" needs to be configured in SecureWebSocketInboundEndpoint.xml file resides under <APIM_HOME>/repository/deployment/server/synapse-configs/default/inbound-endpoints directory in order to invoke WSS based WebSocket API. This step is missing in the documentation [1].
[1] https://docs.wso2.com/display/AM260/Create+a+WebSocket+API

The latest it is already there. Hence closing the issue


Resize Chart - Responsiveness

Hi,
Does CircosJS has resize chart method/functionality?. I would like to resize the chart according to the window resize. Is there anything which is in built already?. If not, please let me know if anyone has done anything similar. Thanks.

@nicgirault I was able to achieve the functionality almost close to auto resize with window using by subscribing to windows resize event and changing the inner radius and outer radius.


Update Footer.js

copyright 2020 -> 2021

Is there an easy way to make it just use the current year?

Is there an easy way to make it just use the current year?


Enable code checking

Currently, the checking of the code via checkstyle, sonarqube and owasp is disabled.
It should be enabled and the sources should be brought to compliance

[ ] Checkstyle
[ ] Sonarqube
[x] owasp (#13)

Integrate those into our GitHub actions workflows. It should be run by every commit into every branch.

Sonarqube does not support java 15 with pattern matching instanceof yet (https://jira.sonarsource.com/browse/SONARJAVA-3505)


Left click on Linux does nothing.

Seems that only right click is mapped for Linux. When I left click it does nothing...
I am using the latest Cinnamon on Arch Linux.
Was curious if this was by design, or if there is a way we can get both Left and Right click to bring up the menu! Or at the very least have a way to map a custom action to Left click so we can get it to do something.
I even tested having a window up (generally I have no windows), then clicking the systray icon to see if it would at least foreground the app, but in my experience it does absolutely nothing. =\

I implemented a solution in my fork, just to have it documented.
https://github.com/zquestz/systray/commit/3a26a41ae0858d04dcbc8bd2959a7af218f9f03d
Basically it just allows you to map a custom function to Activate.


How to get offset of a country. because when we pass offset it give correct time for manual lat lng. if we did not pass offset for manual lat lng it give wrong timings. also clarify that offset is necessary for manual location.

Please follow the general troubleshooting steps first:

[x] I've Read the official README v1 or README v2?
[x] I've Read the official API reference?

Question related to which version: [eg: v1, v2, common]
Description
[Describe the question and your situation]

You can try to use the official timezone library: https://pub.dev/packages/timezone

Other useful resources related to the question: https://stackoverflow.com/questions/16086962/how-to-get-a-time-zone-from-a-location-using-latitude-and-longitude-coordinates


Finding the height of the work area.

Is there an example of finding the height of the work area relative to the robot's base_link by lowering the gripper until it touches the work area? The feedback would be provided by either the current sensing in the xarm's joints or through an added force torque sensor. If so, what interfaces in xarm_msgs should I use?
Thanks!
Edit: I found this C++ example of using the external force torque sensor to use force control in the z-direction (https://github.com/xArm-Developer/xArm-CPLUS-SDK/blob/master/example/8003-force_control.cc). Is the purpose of this demo to show that you can push the end effector in the z negative/positive directions when running this example and the robot arm reacts to the applied external forces?

Hi, there is currently no direct way of finding the touch point without a external Force/Torque (FT) sensor. The example code you mentioned is for our integrated FT sensor and would not be functional with other third party sensors. The purpose of the code is just like your description, to maintain a touching force in a specific direction.
If using our integrated FT sensor under ROS environment, there will be a topic publishing the filtered and compensated sensor reading data, which is  /xarm/uf_ftsensor_ext_states. Otherwise, you may use a third party FT sensor and publish the corresponding data by yourself.

We are going to be purchasing the xArm6 with xArm's external Force/Torque (FT) sensor. Are there any examples of finding the touch point with that hardware configuration? Thanks!

You may monitor the data while command the robot to move along z- direction for example, and stop the robot when the Z force data exceed some threshold. Are you going to use ROS2 to realize this function?

Thanks for your response. Yes, we have a lot of experience with ROS2 and MoveIt2, so we will be using it. Your proposed solution makes sense. We were thinking of a similar approach, but I was unsure about the delay in time between when the robot gripper comes into contact with the work surface, the force data is received by a ROS2 node monitoring the force data, and then the command being sent back to the robot arm to stop moving down. Are there any parameters in the xArm6's firmware that we can tune to minimize the wear on the robot joints to account for that time delay? Thanks!

According to your description, I would probably suggest you to try the "impedance control" function with our integrated FT sensor, this function will make the robot tool side behave like a spring while in contact with the obstacle. Please first read (explanations available inside) and test the code from 8002 example of SDK repository, the parameters worth tuning would be the equivalent stiffness (K_pos/K_ori), mass(M)/moment of inertia(J), and the impedance enabling direction (c_axis).  Maybe just tuning the stiffness would be enough, and the M/J parameter should not be too small for stability consideration.
Note: if you are attaching gripper or not just operating toward Z direction, please do a sensor/load compensation first. Run example 8004 or do it in UFACTORY Studio.
Finally, to implement it in ROS2, please follow ReadMe chapter 5.4 to make the proper configuration to enable the relevant services. The services are listed in a file for your reference:
xarm_user_params.yaml.txt
And here are the ros2 service calling examples to realize the behaviors in SDK example 8001-8003:
ft_sensor_ros2_cmd.txt

Thank you for the information. I have successfully installed the force torque sensor on the xarm6 and I'm able to call the ros2 service to enable the force torque sensor with the command: ros2 service call /xarm/ft_sensor_enable xarm_msgs/srv/SetInt16 "{data: 1"}. After issuing the command, I can see valid data being published to /xarm/uf_ftsensor_ext_states and /xarm/uf_ftsensor_raw_states. However, when the FT sensor is enabled, I see the following ERROR output from the ros2_control_node: [ros2_control_node-6] [ERROR] [1694793504.462182933] [UFACTORY.RobotHW]: [<IP_ADDRESS>] Robot State detected! State: 5. This forces the ros2 control software into a mode where it won't send new joint commands to the xarm6. If I restart the initial ros2 launch command (ros2 launch xarm_moveit_config xarm6_moveit_realmove.launch.py robot_ip:=<IP_ADDRESS> report_type:=rich), I can see the valid data being published to FT data topics and I can use MoveIt to send joint commands. Is this a bug in the xarm ros2_control hardware software or am I doing something in the wrong order? Thanks!

Hi, it is true that the ft_sensor_enable will automatically set the robot state to 5, which is some kind of reset state and the arm will not be ready for new command execution until you manually set the state back to 0.  Please upgrade the controller firmware to the latest version and use rosservice call /xarm/set_state 0 to set the robot to READY state after enabling the FT sensor.

Thanks for you help. We have the ft sensor working with the impedance controller now to find the table height.


Add macOS terraform parser

This PR adds support for macOS in terraform parser.

@michalchecinski Would you like to fix that PR by yourself by running the publish steps on macOS? I can also clone your branch and do that by myself if you're short on time :)


Convert Read commands from Word to MkDocs

Need to convert the Read commands form Word to MkDocs.

I converted the Read commands.  Close the issue.


String class doesn't have case folding methods

Hi,
(First of all thanks to all the contributors - Wren is a really classy language to work with 🙂 )
String class doesn't have methods to fold cases like toLowerCase and toUpperCase. I couldn't find it in the document. Can this be implemented, will be helpful.

I put together a Pull Request for .lower to get the convo started over at  https://github.com/wren-lang/wren/pull/1019
Would love your review of it!


fix social icons style

Changed footer styles and footer-js code to fix styles across browsers

This seams smart 🌮


Make leli command more interactive

Quite a big PR here.
Basically to implement what was discussed on issue #53, we have to change our bot implementation radically. I made a proposal before about using a library and I found that this PyTelegram library is the best out there (forget all about my proposal previously; it doesn't work). It's still pretty young though so I wrap it with our own classes etc and I wrote tests for them too so it should be easy if we want to swap with another library later on.
This PR mostly about refactoring our implementation to use the wrapped library. However I also implemented the interactive leli command. Might be better in a separate PR but hey it's done already.
I consider a user's message is a leli command if 1) it starts with /leli (as usual), or 2) it is a reply to TululBot's message asking what to leli. The details can be viewed on commands.py file.
This PR may be daunting but please do review this. Don't hesitate to ask me should you need explanation or reason why I did something this way etc.

Everything looks good for me. I think we do still have a problem about conversation-mode program flow, but we can figure out it later if everything gets bigger.


Produce and consume messages forever

Rather than producing a few messages, consuming them, then waiting
forever, this PR changes the demo application to alternately produce and
consume messages forever.
It also changes the log output to use printf-style messages. They're not
as good for production observability, but they make the output clearer
to first-time users.

Sample output of docker compose up --build --attach demo without schema enforcement enabled:

With schema enforcement enabled:


Cursor not moving to next occurence of asterisk search

Ubuntu 20.04
VSCode Version: 1.54.1
Extension Version 0.0.78
System neovim Version: NVIM v0.5.0-dev+925-g00f60c2ce
This occurred in a larger cpp file I was working on today, but I've narrowed it down to the following minimal reproducible example:
vscode_neovim_search_bug.txt
Pressing * while the cursor is over the word test should move the cursor to the next occurrence, continuing from the top or bottom of the file if necessary.
However, after a few presses of * (or n) to get the next occurrence, the cursor doesn't move and gets stuck on the current occurrence (unless you press up or down). This doesn't happen if you delete any of the lines that follow the second test, or any of the lines that come before the first test. Likewise, if you remove the four spaces before either test, the search function works normally again.

I couldn't open your example file but I think I am experiencing the same problem. It also happens when jumping to a line number using :LINE_NUM in normal mode.

I'm encountering the same issue with n (or N for reverse) for both asterisk * and traditional forward / search.
Here's a quick video demo. In this example I'm cycling between 4 occurrences of target using the n key. The cursor fails to update on the 3rd match, but successfully updates on the 4th match.
https://user-images.githubusercontent.com/3578681/123525262-2a619f00-d684-11eb-8de8-7156f7ec7743.mp4
Here's the text file for reproducing this issue: temp.txt
I'm running with an empty ~/.config/nvim/init.vim for this test case.
Version info:

vscode 1.57.1
vscode nvim extension: 0.0.82
nvim: v0.5.0-dev+1372-g056c464e8

Same issue here. It happen sometime not always.

Likely a duplicate of #740


Businessrules Referral Prescription V0.9

just for reference:
Business Rules ReferralPrescription V0.9.docx

superseded by #88


Running README example returns different result

I'm on pyschedule v0.2.31 and python v3.5.2. When I run the README example (from my terminal, I run python3, then copy+paste from the README), I get the following solution:
[(clean, Alice, 0, 3), (wash, Bob, 1, 3), (cook, Alice, 5, 6)]

This doesn't match what is listed on the homepage and I'm not sure where else to start. I'm sure I'm missing some other useful information, so let me know what else I can/should provide!
Also, sorry if this duplicates #2 but (i) I didn't want to comment on a closed issue and (ii) it's a different underlying issue.

Sorry for the late reply, the README was indeed not up-to-date, the completion time cost used to be automatically set to 1, but this was removed since this slighly decreases the performance in case it is not needed. But with the single line S.use_flowtime_objective() this can be set manually to ensure that all tasks are pushed to the "left".


Make a bright version of musk rose shader.

          hey. its me again. if you can please make an 80% or 60% brightness musk rose shader if you can. pleaseee. and when I use night vision potion it does not work while I use the shader also....

Originally posted by @hirun1228 in https://github.com/Rinloid/musk_rose_rd/issues/4#issuecomment-1589902251

I won't.  I'm really comfortable with the current brightness.  Night vision also is not supported since the shader don't use the vanilla lightmap sampler.

😭😭😭😭😭😭😭😭😭😭😭😭😭😭😭


Does not seem to work

Tried very hard to make this work, but

Can not download from github
copied file individually - does not work
After hours and fixing here and there
This is the error I get

Error in check_input(x) :
Input must be a character vector of any length or a list of character
vectors, each of which has a length of 1.
In addition: Warning message:
'unnest_tokens_' is deprecated.
Use 'unnest_tokens' instead.


Image Link's are completely mindblowing

They come through to IRC as fully markdown defined:
[![preview-3.jpg](https://files.gitter.im/Team/Blah/o1NT/thumb/preview-3.jpg)](https://files.gitter.im/Team/Blah/o1NT/preview-3.jpg)

Because of the first link "never ending", IRC clients can't properly make it easily clickable ;(

Duplicate of https://github.com/gitterHQ/irc-bridge/issues/44


Improve explore page

Is your feature request related to a problem? Please describe.
When you search for something in the explore navigation, then switch to the main navigation and return to explore again after searching for something in the explore navigation. It is likely that you will face a recycle view that will have some bugs in it.
Describe the solution you'd like
I will make it work perfectly will make sure that everything works perfectly. I'll go through the code, I'm confident that I'll fix this. I'll also make the search bar more user-friendly.

@Aditya-gupta99 Thank you for pointing this out. Please attach screenshots for comparision after you complete

Closed. Added in #123


[BUG]: Long production DFP training times

Version
23.01
Which installation method(s) does this occur on?
Source
Describe the bug.
We are experiencing long training times with our production DFP pipeline. There appears to be an issue with the dfencoder AutoEncoder not fully utilizing our GPU. We're seeing about two hours on first epoch with a dataset that looks like this:

7 categorical variables with cardinality of: 854, 17, 17, 989, 989, 67, and 7455.
800 binary features.
100-200M rows of data in total

Minimum reproducible example
No response
Relevant log output
No response
Full env printout
No response
Other/Misc.
No response
Code of Conduct

[X] I agree to follow Morpheus' Code of Conduct
[X] I have searched the open bugs and have found no duplicates for this bug report

Confirmed the low GPU utilization during training. Most of the training time is spent during the post-processing phase of the fit call, specifically in get_anomaly_score_losses.  By default, get_anomaly_score_losses uses entire training dataframe for calculating loss statistics. By setting use_val_for_loss_stats=True in fit call, we can use much smaller validation set instead. This resulted in 2.5x speedup in my training run using validation_size=0.1.
For added speedup, get_anomaly_score_losses in autoencoder.py` can be updated to use self.eval_batch_size instead of self.batch_size (#861). This will default to use 1024 instead of 512. This resulted in 3.2x speedup when combined with use_val_for_loss_stats=True .

@efajardo-nv should this still be open? Just want to make sure #861 didn't close this out.

@jarmak-nv yes this issue is still ongoing. #861 did not provide enough training improvement. waiting for feedback on the following suggestions:

training single batch using new DFPFileBatcherStage.period == None option provided in PR #912 (instead of multiple incremental models).
downsampling using improved sampling options also provided in PR #912 to reduce training data size
larger training batch size

Got it, thanks for the update!
(Now that I'm fully in the issue instead of from the project view I see the mention of other issues in the timeline. Not the first time the lack of timeline from the project view has gotten me, and probably not the last. Apologies!)

After incorporating all the above suggestions, their training time still was not significantly improved. It was also found that training outside of the pipeline (i.e. using dfencoder directly) is around 10x faster compared to running inside the pipeline.
I was able to reproduce this discrepancy using 23.03 with the 23.07 updates corresponding to the above fixes. The discrepancy appears to go away when I update to 23.07.  Morpheus update is in progress now. Waiting on feedback

Issue resolved by migrating to 23.07 environment and pipeline.


Editor app beta version not set in macOS package info.plist

Godot version
v4.0.beta5.official [89a33d28f]
System information
macOS 13.0.1
Issue description
In macOS app bundles the application version used by macOS get info, picking the latest version of the app, crash reports etc is set in the info.plist file in the app bundle.
There are two keys for this: CFBundleShortVersionString (usually used for the human readable displayed part, like "4.0.beta5") and CFBundleVersion (either used for a longer full version like "4.0.beta5.official [89a33d28f]" or just a build number "89a33d28f")
The possible bug is that currently they're both just set to 4.0 for the betas, so for example crash reports won't show what beta they are and you can't see which beta version the app is on macOS without launching it. The full version info to populate these fields obviously exists, but the fields are apparently not populated automatically in the current build process as they should. Or maybe they are, but the current script cuts away the useful .betaX part of the version string and which just leaves 4.0 as the version.
Steps to reproduce
Inspect the Editor app bundle on macOS for one on the 4.0 betas
Minimal reproduction project
Just download the macOS editor

Currently, Info.plist for the editor is not generated during the build and should be updated manually - https://github.com/godotengine/godot/blob/master/misc/dist/macos_tools.app/Contents/Info.plist

Maybe that could be automated in some way to save some manual work for each release and reduce the risk of forgetting it?

While this might seem like just a cosmetic thing and helpful for debugging it also has the consequence that macOS can't tell multiple installations of Godot apart if you have several versions of the Godot editor. This leads to for example the system not using the most recent Godot version to open double clicked godot project files and also sometimes referring to older versions when granting specific system permissions to Godot (for example needed for some export operations).


GPT-4 Models have large differences in price

I think that we should allow some further customization of which models that a user actually wants to use when translating, because the regular gpt-4 model is significantly more expensive than the turbo versions of the same models.
But all of these models are grouped together as choices to pull from when translating content. I ran a short test of 75 paragraphs and the price disparity is pretty shocking:

Vanilla gpt4

Turbo gpt4

Proposed solution
I think that it would be a good idea to expose gpt4-turbo as well as all-gpt4 as model options and include some documentation that explains this, because if I wasn't paying attention I could have accidentally significantly overpaid.
Another option would be just to remove vanilla gpt-4 by default since the turbo models have incredibly high limits on even level 1 accounts now:

I'm open to discussing better options here, but I do think that at the very least some documentation updates should be made to make it clear that both turbo and regular models are grouped together like this. Happy to hop in and help if there is some consensus on a good structure for this.
I would just hop in and write the code, but probably worth some discussion since this would involve some changes to the external API.

yes you are right we can fix one specific model, PR welcome for that.
Thank you very much

Perfect. I will take a look at adding this code tonight.

Resolved with that merge


How can I count the number of shaking times

No, I don't think there's an optimal way to do that. Since Android (different variants count too) and iOS have their own algorithm to count "actual" shaking times.


test(transformer/class-properties): override test output for _super in class constructor

Our output for this test differs from Babel, because we handle super() in class constructors differently, but our output is valid.

#7717
#7729  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

I think I know this particular one better than anyone else, so merging without review.

Merge activity

Dec 8, 12:54 PM EST: The merge label '0-merge' was detected. This PR will be added to the Graphite merge queue once it meets the requirements.


Harmony Remote component stops Home Assistant from loading if hub is not reachable

Make sure you are running the latest version of Home Assistant before reporting an issue.
You should only file an issue if you found a bug. Feature and enhancement requests should go in the Feature Requests section of our community forum:
Home Assistant release (hass --version):
0.36.1
Python release (python3 --version):
3.5
Component/platform:
remote.harmony
Description of problem:
If the hub is not online, home assistant will not start
Expected:
Home Assistant should start and log an error
Problem-relevant configuration.yaml entries and steps to reproduce:
remote:
  - platform: harmony
    name: Basement
    host: <IP_ADDRESS>
    activity: Watch TV

Traceback (if applicable):
17-01-19 17:53:52 INFO (MainThread) [homeassistant.components.remote] Setting up remote.harmony
17-01-19 17:53:52 DEBUG (Thread-11) [homeassistant.components.remote.harmony] Loading Harmony platform: Basement
17-01-19 17:53:52 DEBUG (Thread-11) [homeassistant.components.remote.harmony] Calling pyharmony.ha_get_token for remote at: <IP_ADDRESS>:5222

I can confirm I'm having the same problem still, in version 0.37.1
I also have an easier way to reproduce: Just set the ip where it looks for the hub to something that isn't the hub. (I ran into this problem because my router reset and lost my static leases)

Same issue here, Same wrong IP messing things up.  I only got to "Setting up remote.harmony" before it hung.

I know some components (like ecobee and the automatic part of the plex component) pop up a configurator when they aren't correct, is that approach a good candidate here too?  To alert the user there's a problem and a common solution is check the IP address

The best part is that there is no indication this is the problem. There's no error message in the log or systemctl status output. I only know this is it by guessing and checking

Working on this.

Problem still exists after Hassbian update to 0.46.
In case of harmony hub has no power connection following error will be written on startup:
2017-06-04 23:28:59 ERROR (MainThread) [homeassistant.components.remote] Setup of platform harmony is taking longer than 60 seconds. Startup will proceed without waiting any longer.
But Home Assistant do not start.

@Nex81, that seems to be fixed by #7970 which was released with 0.47.


Futures index does not return Saturday and Sunday data

There are futures indexes that trade on Saturday and Sunday that are tracked by Yahoo Finance. These indexes are denoted with tickers such as:
ES=F is ^GSPC futures
RTY=F is Russell 2000 futures
MNQ=F is NASDAQ futures
The yfinance package does not return the Saturday and Sunday values for these kinds of tickers. I've been looking for where in the yfinance code the Saturday and Sunday values are dropped but I can't find it. Perhaps the data is not returned in the first place.

Maybe Yahoo simply isn't returning the data? Easy to check, just edit code to print out the data that yfinance receives.

I did. It's not returning it. I am trying to figure out where the data is stored on the yahoo finance web page because it is showing on the actual web page accessed through a web browser. I could be wrong about this though. I would say my programming skills are between novice and intermediate.

Try different intervals

1h interval returns Sunday data.
Please show screenshot or url to Yahoo showing Sunday daily data because I don't see it.

I figured it out. The dates being returned are off. For instance; 5/19/22 (Thursday) is being recorded in the day 5/20/22 (Friday). So Sunday is being recorded as Monday in the dataframe.

Ah, timezones. I think I've fixed that but need a guinea pig to test it. Can you volunteer?
Grab my fork: https://github.com/ValueRaider/yfinance/tree/fix/timezone
And load like:
import sys
sys.path.insert(0,"path/to/my/yfinance")
...

Okay, one sec

Maybe yahoo has code that is fixing this bug on the user side instead of the server side because this would only happen for a few securities

I'm getting a match between Yahoo and yfinance for May 23rd. Are you sure you're loading the fork?

I used pip install git+https://github.com/ValueRaider/yfinance.git#egg=yfinance to install it, I believe this is correct. I will try installing it under a different name to make sure it isn't using a cached version

Oh, sorry about that, thank you for being patient, one sec

I am still getting the original repo when I go to the link. What is the .git link for the new repo?

sigh
The fix is in a branch in my fork. The url goes direct to it, you can't miss it.

I'm on this page: https://github.com/ValueRaider/yfinance/tree/fix/timezone and I go to the drop down menu named code and get the .git link https://github.com/ValueRaider/yfinance.git but that isn't the correct link.

Yes it is. You download the repository, then switch to the branch.

Its not working for me. I am still getting the 20th and 23rd instead of the 19th and 22nd.

I need to be convinced that you are loading my branch. Run this:
...
import yfinance as yf
if "site-packages" in yf.__path__[0]:
	print("Not loading forked repository, but: "+yf.__path__[0])
	quit()

import os
f = open(os.path.join(yf.__path__[0], "base.py"), 'r').read().find("cache_lookup_tkr_tz")
if f == -1:
	print("Not loading my fork")

df = yf.Ticker("ES=F").history(start="2022-05-19",end="2022-05-20", interval="1d")
print(df[["Close"]]) # If date doesn't have time then not my branch

Yahoo doesn't have weekend data.


feat(ast): serialize StringLiterals to ESTree without raw

#7263  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @ottomated and the rest of your teammates on  Graphite


Fix for iOS 9 Banner not taking navigation bars into account properly

This fix is based in part @Loadex's  feature/newTopLayout branch.
This issue appears to be caused by a bug in iOS 9?
The statement realtViewController.topLayoutGuide seems to go away at some point after we set it here. To be clear it is not nil at run time when [... constraintWithItem:realViewController.topLayoutGuide ...] is called however when I bring this up in Reveal App the constraint is labeled as ???.bottom = MBLMessageBannerView:0x... and the First Item is Unknown.
After a few hours banging my head against a wall this was the best I could come up with and it seems to work but I am skeptical as to how robust this really is.

Thanks for the pull request. This can be used as a quickfix is someone really need a fast fix. I will take a look this weekend to see how this can be improved.

Yes, I agree this is not a proper fix long term, and is more of a quick hack to keep things moving while the proper solution is found.

Was a long term fix ever implemented? Also MessageBanner really doesn't like use_frameworks! in CocoaPods.


#184581188 A User Should be able to list all Items

What does this PR do?
This PR help A user/Seller to get a list of product in his/her collection
Description of Task to be completed?

Creating route to get items in seller collection
protecting route
testing route
document route

How should this be manually tested?

Clone this repo git clone https://github.com/atlp-rwanda/e-comm-team-amigos-bn
Switch your working directory cd e-comm-team-amigos-bn
Create an .env file and update it with .env-example
Install all the dependencies by running npm install
make sure you have testing tool like Postman
Then run the project by typing npm run dev
you should create a user with a role of vendor

Endpoints to be tested:

/user/login/
/user/otp/
/product/create/
/product/collection/

Any background context you want to provide?

Have Node Js installed in your local machine

What are the relevant pivotal tracker/Trello stories?
Story ID: #184581188
Screenshots (if appropriate)
Questions:

Update your readme file, provide documented endpoint screenshot and passing test screenshot if there are available. and remember to resolve conflict from this branch.


[Snyk] Fix for 1 vulnerabilities

This PR was automatically created by Snyk using the credentials of a real user.Snyk has created this PR to fix one or more vulnerable packages in the `npm` dependencies of this project.
Changes included in this PR

Changes to the following files to upgrade the vulnerable dependencies to a fixed version:

packages/jmix-front-generator/package.json
packages/jmix-front-generator/package-lock.json

Vulnerabilities that will be fixed
With an upgrade:

Severity
Priority Score (*)
Issue
Breaking Change
Exploit Maturity

661/1000  Why? Recently disclosed, Has a fix available, CVSS 7.5
Missing Release of Resource after Effective Lifetime SNYK-JS-INFLIGHT-6095116
Yes
No Known Exploit

(*) Note that the real score may have changed since the PR was raised.

Commit messages

Package name: yeoman-environment
    The new version differs by 236 commits.

8ead0c2 4.0.0
1250965 try to fix lint errors
c7028e1 fix npm audit problems and adjusts
eae5a81 4.0.0-rc.0
e668ddc bump dependencies
8d4147e bump yeoman-generator-6
47cdc8f fix eslint error
7f104a7 integrate yeoman-generator (2, 4 and 5) tests.
2569915 rework commitSharedFsTask for mem-fs@4
adcda06 4.0.0-beta.6
f8f26a3 bump dependencies and allow mem-fs@4
100c609 make logCwd public
12cb28f remove mem-fs-editor instance from environment
3877e04 4.0.0-beta.5
bd8e8e6 update conflicter (#492)
c88cb66 drop node 16 support
9bb755e fix conflicter's ignore option
367a340 Bump actions/checkout from 3 to 4 (#490)
ecac892 4.0.0-beta.4
b3ffea0 bump development dependencies.
960d627 increase stale for PRs
9323a1f allow dependabot to update typescript
b28085d create package-lock.json
5ed4c37 drop tui-jsdoc-template dependency

See the full diff

Package name: yeoman-generator
    The new version differs by 144 commits.

5e59844 6.0.0
4296f50 6.0.0-rc.6
b224faa fix typo
7f1f8e1 allow to customize queueTransformStream priority
0eb085f 6.0.0-rc.5
232587e Run own or immediately extended beforeQueue (#1475)
1938e71 Bump actions/checkout from 3 to 4 (#1473)
2d8b8b1 Bump execa from 7.1.1 to 8.0.1 (#1469)
5b729fe Bump read-pkg-up from 9.1.0 to 10.1.0 (#1470)
465aea2 6.0.0-rc.4
df635f1 Fix composeLocallyWithOptions with a factory
7e6b0c0 6.0.0-rc.3
5291e9c allow to ignore 'This Generator is empty' error
454b3ec Bump c8 from 7.13.0 to 8.0.0 (#1457)
077c6e9 6.0.0-rc.2
02e6785 don't emit error at environment, environment's queueTask to handle failures.
a75dbf8 6.0.0-rc.1
7fa5597 replace lodash with lodash-es
fbae307 use TestAdapter from @ yeoman/adapter
580ba5c add spawn/spawnSync
66fd6be bump got package
9428e88 Revert "disable spawn-command test due to got package bug."
1d2889d make test compatible with environment 4
1a11e03 6.0.0-rc.0

See the full diff

Check the changes in this PR to ensure they won't cause issues with your project.

Note: You are seeing this because you or someone else with access to this repository has authorized Snyk to open fix PRs.
For more information:
🧐 View latest project report
🛠 Adjust project settings
📚 Read more about Snyk's upgrade and patch logic

Learn how to fix vulnerabilities with free interactive lessons:
🦉 Learn about vulnerability in an interactive lesson of Snyk Learn.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Segmentation fault when evaluating some samples

Hi, I was trying to reproduce the results following the script you provided. However, I faced segmentation issues when doing the EVALB evaluation. I think the issue has to do with predicted parses, EVALB fails for some of the instances while not for others.
Appreciate it if you could explain or help solve this issue. Thank you!

Occasionally we saw this issue with EVALB early in training, before the parser learned to make decent trees.  We stop getting seg faults later on when the predicted trees become more accurate.
Could this be the case here? If not, we'd love to investigate this issue further, especially if you have any other relevant information on it.
Thanks!

I find that this is because that evalb limits the number of parentheses in a sentence. (at line 62 of evalb.c) You can change that macro macro definition to solve this problem.
BTW.  In my dataset, there are some url tokens whose lengths exceed the macro MAX_WORD_LEN, which also causes segmentation fault.


Create a "one click" dev env for Jobs

This included documentation about what other things are needed I. E. Credentials for aws etc.
Should also include some dummy data.

This has been now better documented. It is not one click and will not be any time soon because of somewhat complicated dependencies on external services that would be hard/expensive to mock out.


Merge ecoacoustics-datamodule fixes into ecoacoustics-dm-2

As I understand it, this can be closed (feel free to re-open if not).


Error: Failed to parse array element

Hi again,
One more load error for you.  I'm on v0.6.1-rc2.  Thanks again!
Error: Failed to parse array element
    at error (pdf-lib.js:8569)
    at parseArray (pdf-lib.js:46691)
    at parseDict (pdf-lib.js:46635)
    at parseDictOrStream (pdf-lib.js:46908)
    at parseIndirectObj (pdf-lib.js:46947)
    at parseBodySection (pdf-lib.js:47174)
    at parseDocument (pdf-lib.js:47226)
    at PDFParser.parse (pdf-lib.js:47257)
    at Function.PDFDocumentFactory.load (pdf-lib.js:47379)

This is a doc (1).pdf

I took a look at this document as well. It is corrupted in a similar way as what I described in https://github.com/Hopding/pdf-lib/issues/68. Since these errors stem from the same issue (a corrupted document), I'm going to close this issue so we can centralize conversation in https://github.com/Hopding/pdf-lib/issues/68.

Hi @Hopding ,
You're correct that this isn't causing a problem in production.  For reference, it was created with Adobe Acrobat pro on a windows machine while I was doing some testing.


Streamline Your Contributions: Essential Guidelines for Effective Pull Requests

Important
Before making a Pull Request (PR), please ensure you follow these steps:

Raise an Issue First: Always raise an issue before submitting a PR. This helps us track and discuss the problem or feature request before any code changes are made. PRs submitted without a corresponding issue will not be accepted.

Proper Documentation: When making a PR, include comprehensive documentation. This should cover:

A clear description of the changes made.
Screenshots or demo videos showcasing the functionality or bug fix.
Any relevant details that help reviewers understand the context and purpose of the PR.

Home Page Issues: Do not raise any issues regarding the home page or front page. These areas are currently under special management and are not open for general contributions.

Following these guidelines will streamline the review process and ensure that your contributions are effectively integrated into the project.
Thank you for your cooperation!

@iamrahulmahato You have to add level labels in the issues as well, later if you find that the work should have another level, then you can change it and upgrade or degrade, for the time being, assign level as per the complexity of the task mentioned/described in the issue
-GSSOC Team

@vansh-codes-dev Ok, will do. Thanks for the guidance!

Please assign it to me .


prohibit e.g. np.ndarray as parameters

Either prohibit or find a good way to ensure proper znjson conversion excluding pickle.

relevant code in IPSuite https://github.com/zincware/IPSuite/blob/13814726a362f5135352f501e8fa26aad77c2a94/ipsuite/configuration_comparison/base.py#L146


Rename repo to xanzy/gitlab

The package name is gitlab while the repo name is go-gitlab. This works, but results in confusing usage:
import (
	"github.com/xanzy/go-gitlab"
)

func foo() {
	git := gitlab.NewClient(nil, "yourtokengoeshere")
}

Where is gitlab being declared or imported? This is an extreme case, but if all packages did this, complex programs would be a mess. You could go around this as a user by explicitly importing gitlab "github.com/xanzy/go-gitlab", but that's really unnecessary.
Moreover, the Go authors discourage calling packages or programs go-* simply because they are written in Go. You may think that calling your repo gitlab might confuse people with the GitLab reo itself, but it's easy to tell apart since one is a Ruby app and another a Go package.

@mvdan I see your point, yet I think this is pretty common. That doesn't make it the right pattern of course, but it's also not something a Go developer should be puzzled about.
Also I know about the calling packages or programs go-* simply because they are written in Go is discouraged. But in this case I believe the current name makes a lot of sense as it tells you all you need to know... It is a GitLab client for the Go language.
If you look at all the other repo names for GitLab API clients (here: https://about.gitlab.com/applications/) you'll see that quite a few have similar names: python-gitlab, backbone-gitlab, php-gitlab-api, java-gitlab-api.
Lastly I think that changing the import URL at this point in time isn't a user friendly thing to do. So long story short, I believe we should not change this and will close this issue as a won't fix.
Thanks!

Fair enough. I don't see why you think it wouldn't be user friendly, though - github would redirect the old path to the new one. So all current imports, clones and links would continue to work.


'uniform = FALSE' does not work

using the example from the help:
library(randomForest) ; library(edarf)
data(iris); data(swiss)
`## classification` `fit = randomForest(Species ~ ., iris)`
## WORKING ##
pd = partial_dependence(fit, c("Sepal.Width", "Sepal.Length"), data = iris[, -ncol(iris)])
## NOT WORKING ##
pd = partial_dependence(fit, c("Sepal.Width", "Sepal.Length"), data = iris[, -ncol(iris)],  ,uniform = FALSE)
## Error in partial_dependence(fit, c("Sepal.Width", "Sepal.Length"), data = iris[,  :
##   Assertion on 'y' failed: Must be of type 'data.frame', not 'double'.
Cheers, thanks for having written this useful package !

this is a versioning bug with mmpf which edarf relies on. working on it now

if you use the github version of mmpf it should work in the mean time. i submitted an updated version of mmpf to cran


Need search option for matched users as dynamic option

As a vsinder tester Need search option for matched users  as dynamic option ; )

the search can be like texting input (default field "Search..")

I Support this ideal!


A successfully removed container occasionally prevents a container with the same name from starting

Output of docker version:
Client:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   b9f10c9
 Built:        Wed Jun  1 21:23:11 2016
 OS/Arch:      linux/amd64

Server:
 Version:      1.11.2
 API version:  1.23
 Go version:   go1.5.4
 Git commit:   b9f10c9
 Built:        Wed Jun  1 21:23:11 2016
 OS/Arch:      linux/amd64

Output of docker info:
# docker info
Containers: 2
 Running: 2
 Paused: 0
 Stopped: 0
Images: 27
Server Version: 1.11.2
Storage Driver: devicemapper
 Pool Name: docker-202:2-25410788-pool
 Pool Blocksize: 65.54 kB
 Base Device Size: 10.74 GB
 Backing Filesystem: xfs
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data Space Used: 3.781 GB
 Data Space Total: 107.4 GB
 Data Space Available: 14.79 GB
 Metadata Space Used: 6.427 MB
 Metadata Space Total: 2.147 GB
 Metadata Space Available: 2.141 GB
 Udev Sync Supported: true
 Deferred Removal Enabled: false
 Deferred Deletion Enabled: false
 Deferred Deleted Device Count: 0
 Data loop file: /var/lib/docker/devicemapper/devicemapper/data
 WARNING: Usage of loopback devices is strongly discouraged for production use. Either use `--storage-opt dm.thinpooldev` or use `--storage-opt dm.no_warn_on_loop_devices=true` to suppress this warning.
 Metadata loop file: /var/lib/docker/devicemapper/devicemapper/metadata
 Library Version: 1.02.107-RHEL7 (2016-06-09)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge null host
Kernel Version: 3.10.0-327.22.2.el7.x86_64
Operating System: Red Hat Enterprise Linux Server 7.2 (Maipo)
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.451 GiB
Name: ip-172-31-24-78.us-west-1.compute.internal
ID: RC5P:NONS:4W7K:F5EO:ZC55:6W2Q:CTEM:VUXK:GPHP:M7BR:CPTI:M6VZ
Docker Root Dir: /var/lib/docker
Debug mode (client): false
Debug mode (server): false
Registry: https://index.docker.io/v1/
WARNING: bridge-nf-call-iptables is disabled
WARNING: bridge-nf-call-ip6tables is disabled

Additional environment details (AWS, VirtualBox, physical, etc.):
The host is on AWS, RHEL 7, AMI RHEL-7.2_HVM_GA-20151112-x86_64-1-Hourly2-GP2 (ami-2051294a)
Steps to reproduce the issue:
We have 3 containers, replicated, replicated-ui, replicated-operator. The containers are started via standard systemd unit files.
During a cleanup procedure, we have a script that first stops the 3 systemd units (the containers get killed), and then we remove them with:
docker rm -f replicated replicated-ui replicated-operator

The command completes without errors.
Immediately after, we start again the systemd units. Normally, all the 3 containers will come up without any problem, but roughly once every 20-30 times the replicated-ui unit will refuse to start. In journalctl we see the error:
Jul 15 19:15:57 ip-172-31-24-78.us-west-1.compute.internal docker[18080]: /usr/bin/docker: Error response from daemon: Conflict. The name "/replicated-ui" is already in use by container be8c1e37264c533b69c6bf68a608b15cdc92da01b8c5ff3a544a6ef86c298275. You have to remove (or rename) that container to be able to reuse that name..

Despite the error, there are no replicated-ui containers in use:
# sudo docker ps -a
CONTAINER ID        IMAGE                                         COMMAND                  CREATED             STATUS              PORTS                              NAMES
9b055c9928a5        quay.io/replicated/replicated-operator:beta   "/usr/bin/replicated-"   2 minutes ago       Up 2 minutes                                           replicated-operator
8c32dd120dd7        quay.io/replicated/replicated:beta            "entrypoint.sh -d"       2 minutes ago       Up 2 minutes        <IP_ADDRESS>:9874-9880->9874-9880/tcp   replicated

The error is not systemd-specific, since also trying to start a container manually fails with the same error:
# docker run -it --name replicated-ui --rm busybox
docker: Error response from daemon: Conflict. The name "/replicated-ui" is already in use by container be8c1e37264c533b69c6bf68a608b15cdc92da01b8c5ff3a544a6ef86c298275. You have to remove (or rename) that container to be able to reuse that name..

The API itself doesn't show any trace of the be8c1e37 container (which was removed during the previous docker rm command):
# echo -e "GET /containers/json?all=1 HTTP/1.0\r\n" | nc -U /var/run/docker.sock
HTTP/1.0 200 OK
Content-Type: application/json
Server: Docker/1.11.2 (linux)
Date: Fri, 15 Jul 2016 23:17:42 GMT

[{"Id":"9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a","Names":["/replicated-operator"],"Image":"quay.io/replicated/replicated-operator:beta","ImageID":"sha256:43921c2ed20934b9abe3ab7767b49a4ccad59f78139ed89983c526a05d1125cc","Command":"/usr/bin/replicated-operator","Created":1468624422,"Ports":[],"Labels":{},"State":"running","Status":"Up 3 minutes","HostConfig":{"NetworkMode":"default"},"NetworkSettings":{"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"","EndpointID":"2f4f0879b975cf2c4841910a19515c22d84b570ec7d0285645b8b92edbab5675","Gateway":"<IP_ADDRESS>","IPAddress":"<IP_ADDRESS>","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:03"}}},"Mounts":[{"Source":"/etc/system-release","Destination":"/host/etc/system-release","Mode":"ro","RW":false,"Propagation":"rprivate"},{"Source":"/","Destination":"/replicated/host","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/var/lib/replicated-operator","Destination":"/var/lib/replicated-operator","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/var/run/docker.sock","Destination":"/host/var/run/docker.sock","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/etc/docker/certs.d","Destination":"/host/etc/docker/certs.d","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/proc","Destination":"/host/proc","Mode":"ro","RW":false,"Propagation":"rprivate"},{"Source":"/etc/os-release","Destination":"/host/etc/os-release","Mode":"ro","RW":false,"Propagation":"rprivate"}]},{"Id":"8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b","Names":["/replicated"],"Image":"quay.io/replicated/replicated:beta","ImageID":"sha256:14d1d031ce7518316e8f7fb28e45e53731d5debab9500e0a5ccb6a9f125ae533","Command":"entrypoint.sh -d","Created":1468624419,"Ports":[{"IP":"<IP_ADDRESS>","PrivatePort":9875,"PublicPort":9875,"Type":"tcp"},{"IP":"<IP_ADDRESS>","PrivatePort":9874,"PublicPort":9874,"Type":"tcp"},{"IP":"<IP_ADDRESS>","PrivatePort":9880,"PublicPort":9880,"Type":"tcp"},{"IP":"<IP_ADDRESS>","PrivatePort":9879,"PublicPort":9879,"Type":"tcp"},{"IP":"<IP_ADDRESS>","PrivatePort":9878,"PublicPort":9878,"Type":"tcp"},{"IP":"<IP_ADDRESS>","PrivatePort":9877,"PublicPort":9877,"Type":"tcp"},{"IP":"<IP_ADDRESS>","PrivatePort":9876,"PublicPort":9876,"Type":"tcp"}],"Labels":{},"State":"running","Status":"Up 4 minutes","HostConfig":{"NetworkMode":"default"},"NetworkSettings":{"Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"","EndpointID":"80455e65d64630f421c6c53f8ce25de0cdb28559482fa51f9782fe861bd1f7e8","Gateway":"<IP_ADDRESS>","IPAddress":"<IP_ADDRESS>","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:02"}}},"Mounts":[{"Source":"/etc/replicated.conf","Destination":"/etc/replicated.conf","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/var/run/docker.sock","Destination":"/host/var/run/docker.sock","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/etc/system-release","Destination":"/host/etc/system-release","Mode":"ro","RW":false,"Propagation":"rprivate"},{"Source":"/","Destination":"/replicated/host","Mode":"ro","RW":false,"Propagation":"rprivate"},{"Source":"/etc/replicated.alias","Destination":"/etc/replicated.alias","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/var/run/replicated","Destination":"/var/run/replicated","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/var/lib/replicated","Destination":"/var/lib/replicated","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/etc/docker/certs.d","Destination":"/host/etc/docker/certs.d","Mode":"","RW":true,"Propagation":"rprivate"},{"Source":"/proc","Destination":"/host/proc","Mode":"ro","RW":false,"Propagation":"rprivate"},{"Source":"/etc/os-release","Destination":"/host/etc/os-release","Mode":"ro","RW":false,"Propagation":"rprivate"}]}]

This is also the listing of the /var/lib/docker/containers directory, there's no evidence of such container:
# find /var/lib/docker/containers/
/var/lib/docker/containers/
/var/lib/docker/containers/a1e7847a03991d6b70bb9a4cb725bb4e8483edf81c6c24508d77fa3f51f46e5b
/var/lib/docker/containers/a1e7847a03991d6b70bb9a4cb725bb4e8483edf81c6c24508d77fa3f51f46e5b/shm
/var/lib/docker/containers/744717db78fb6a74dd87ca13090ad20c38f4197698f1a7443245ef4f62b9ab1b
/var/lib/docker/containers/744717db78fb6a74dd87ca13090ad20c38f4197698f1a7443245ef4f62b9ab1b/shm
/var/lib/docker/containers/6d34c520f2130f3daa67061caf5d3236d4cefdd62d64e0251a5102f0d0e0ad6f
/var/lib/docker/containers/6d34c520f2130f3daa67061caf5d3236d4cefdd62d64e0251a5102f0d0e0ad6f/shm
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b/hostconfig.json
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b/config.v2.json
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b/hosts
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b/resolv.conf
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b/resolv.conf.hash
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b/hostname
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b/shm
/var/lib/docker/containers/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b/8c32dd120dd74136927927ac9ef341329b93ed1e80abd22c3ad1d8f8399de80b-json.log
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a/hostconfig.json
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a/config.v2.json
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a/hosts
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a/resolv.conf
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a/resolv.conf.hash
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a/hostname
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a/shm
/var/lib/docker/containers/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a/9b055c9928a5c06762df79390abb646cdd53b0471b6dec5600e0e5a82c8cb77a-json.log

Restarting the Docker daemon almost always solves the problem.
As an additional information, these containers are sharing some parts of the host file system (e.g. -v /:/host:ro, so during removal we consistently see:
Jul 15 19:46:28 ip-172-31-24-78.us-west-1.compute.internal kernel: device-mapper: ioctl: unable to remove open device docker-202:2-25410788-872342a6efad95f336cc17a07aa34f0af3061a0815d68c41bce7ff5e4bc663cb
Jul 15 19:46:28 ip-172-31-24-78.us-west-1.compute.internal kernel: device-mapper: ioctl: unable to remove open device docker-202:2-25410788-872342a6efad95f336cc17a07aa34f0af3061a0815d68c41bce7ff5e4bc663cb
Jul 15 19:46:28 ip-172-31-24-78.us-west-1.compute.internal kernel: device-mapper: ioctl: unable to remove open device docker-202:2-25410788-872342a6efad95f336cc17a07aa34f0af3061a0815d68c41bce7ff5e4bc663cb
Jul 15 19:46:28 ip-172-31-24-78.us-west-1.compute.internal kernel: device-mapper: ioctl: unable to remove open device docker-202:2-25410788-872342a6efad95f336cc17a07aa34f0af3061a0815d68c41bce7ff5e4bc663cb
Jul 15 19:46:28 ip-172-31-24-78.us-west-1.compute.internal kernel: device-mapper: ioctl: unable to remove open device docker-202:2-25410788-872342a6efad95f336cc17a07aa34f0af3061a0815d68c41bce7ff5e4bc663cb
Jul 15 19:46:28 ip-172-31-24-78.us-west-1.compute.internal kernel: device-mapper: ioctl: unable to remove open device docker-202:2-25410788-872342a6efad95f336cc17a07aa34f0af3061a0815d68c41bce7ff5e4bc663cb

Even though this is clearly not a good practice (it's currently out of our control), Docker shouldn't be affected by this in the way we're seeing it (reporting that a container is running without even showing it in the API).

I have the same issue when using the gitlab-multi-runner, which creates containers for me.
When I triggered an action in this tool that needs to create a fresh container, I got these two log messages:
Jul 28 10:32:31 ci-linux docker[1191]: time="2016-07-28T10:32:31.428850856+02:00" level=error msg="Handler for DELETE /v1.18/containers/runner-03e2d752-project-479-concurrent-3-predefined returned error: No such container: runner-03e2d752-project-479-concurrent-3-predefined"
Jul 28 10:32:31 ci-linux docker[1191]: time="2016-07-28T10:32:31.430649681+02:00" level=error msg="Handler for POST /v1.18/containers/create returned error: Conflict. The name \"/runner-03e2d752-project-479-concurrent-3-predefined\" is already in use by container 11e1546aa8545d23a1e1836f6d2e916008cfcde415320e2007d3cb281417050c. You have to remove (or rename) that container to be able to reuse that name."

I could trigger this behavior repeatedly.
The name and id was always the same. Like @gianlucaborello I could not find any trace of the container 11e15.

Let me close this ticket for now, as it looks like it went stale.


startDontHideSelf 能否支持 launch mode？

Issues Guideline
Following information can help us to resolve the issue faster.

Library version
Support-v4 version
Logs
Screenshots

In addition, we do not accept issues unrelated to Fragmentation.

v1.3.5 :)