Why does shuffling my validation set in Keras change my model's performance?

Why I'm confused:
If I test my model on examples [A, B, C], it will obtain a certain accuracy. If I test the same model on examples [C, B, A], it should obtain the same accuracy. In other words, shuffling the examples shouldn't change my model's accuracy. But that's what seems to be happening below:
Step-by-step:
Here is where I train the model:
model.fit_generator(batches, batches.nb_sample, nb_epoch=1, verbose=2,
                    validation_data=val_batches,
                    nb_val_samples=val_batches.nb_sample)

Here is where I test the model, without shuffling the validation set:
gen = ImageDataGenerator()
results = []
for _ in range(3):
    val_batches = gen.flow_from_directory(path+"valid", batch_size=batch_size*2,
                                          target_size=target_size, shuffle=False)
    result = model.evaluate_generator(val_batches, val_batches.nb_sample)
    results.append(result)

Here are the results (val_loss, val_acc):
[2.8174608421325682, 0.17300000002980231]
[2.8174608421325682, 0.17300000002980231]
[2.8174608421325682, 0.17300000002980231]

Notice that the validation accuracies are the same.
Here is where I test the model, with a shuffled validation set:
results = []
for _ in range(3):
    val_batches = gen.flow_from_directory(path+"valid", batch_size=batch_size*2,
                                          target_size=target_size, shuffle=True)
    result = model.evaluate_generator(val_batches, val_batches.nb_sample)
    results.append(result)

Here are the results (val_loss, val_acc):
[2.8174608802795409, 0.17299999999999999]
[2.8174608554840086, 0.1730000001192093]
[2.8174608268737793, 0.17300000059604645]

Notice that the validation accuracies are inconsistent, despite an unchanged validation set and an unchanged model. What's going on?

evaluate_generator won't go through every single item in your directory unless the number of samples passed to evaluate_generator is exactly equal to your number of images. Also, there may be some images that are dropped if your count isn't an exact multiple of batch size.

Can you confirm that val_batches.nb_sample exactly matches the number of images you expect?
Can you try setting your batch_size to 1 and see if you still see the issue?

Alternatively, if you're using anything like batchnorm, then the differences are expected. If you have different batches, they will have different norms and therefore different results.
If none of the above, it could just be a rounding error or something. The differences are small.
Cheers

Can you confirm that val_batches.nb_sample exactly matches the number of images you expect?

Yes, the nb_sample attribute from the object returned by flow_from_directory gives this number.

Can you try setting your batch_size to 1 and see if you still see the issue?

This worked. The accuracies were the same.
However, when I tried a different exact multiple, the inconsistencies came back. I tried a batch size of 100 with 1000 images.

anything like batchnorm

I'm using VGG16. It doesn't have any batchnorm, but it does have dropout.

If none of the above, it could just be a rounding error or something. The differences are small.

This might be it. But either way, I'm still confused about why shuffle=False seems to prevent the differences while shuffle=True seems to cause them.
Thank you

From StackOverflow:

This is a really interesting problem. The answer is that it's because of that neural networks are using a float32 format which is not so accurate as float64 - the fluctuation like this are simply the realisation of an underflow phenomenon.
It case of your loss - you may notice that the differences are occuring after 7th decimal digit of a fractional part - what is exactly the precision of a float32 format. So - basically - you may assume that all numbers presented in your example are equal in terms of a float32 representation.
- Marcin Możejko

My main response:

Assuming this is an underflow problem, why does setting shuffle=True cause the inconsistency while setting shuffle=False doesn't?

The rest of my response:

Anyway, I think you're right. From Wikipedia: "no more than 9 significant decimal digits can be stored" and I see that for the accuracies, the inconsistencies start at the 10th decimal digit. I'm a little confused about the loss values (first column). Why do the inconsistencies start at the 8th decimal digit there but start at the 10th decimal digit for the accuracies (second column)?

To make this super clear:
Why does setting shuffle=True cause the inconsistency while setting shuffle=False doesn't?

I'm closing this issue because my question is answered. I have a new question, but I'll open a new issue for it.
The question:

Why does shuffling my validation set in Keras change my model's performance?

The answer:

It doesn't. Your model's performance is the same.


Role "postgres" does not exist

I just cloned conda-store @ 0083077, ran docker compose up --build inside conda-store/examples/docker and got the following message about the postgres role not existing:
docker-postgres-1            | 2022-06-08 19:56:28.863 UTC [28] FATAL:  password authentication failed for user "postgres"
docker-postgres-1            | 2022-06-08 19:56:28.863 UTC [28] DETAIL:  Role "postgres" does not exist.
docker-postgres-1            | 	Connection matched pg_hba.conf line 99: "host all all all md5"
docker-conda-store-server-1  | Traceback (most recent call last):
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3280, in _wrap_pool_connect
docker-conda-store-server-1  |     return fn()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 310, in connect
docker-conda-store-server-1  |     return _ConnectionFairy._checkout(self)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout
docker-conda-store-server-1  |     fairy = _ConnectionRecord.checkout(pool)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 476, in checkout
docker-conda-store-server-1  |     rec = pool._do_get()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 256, in _do_get
docker-conda-store-server-1  |     return self._create_connection()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection
docker-conda-store-server-1  |     return _ConnectionRecord(self)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 371, in __init__
docker-conda-store-server-1  |     self.__connect()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 665, in __connect
docker-conda-store-server-1  |     with util.safe_reraise():
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
docker-conda-store-server-1  |     compat.raise_(
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
docker-conda-store-server-1  |     raise exception
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
docker-conda-store-server-1  |     self.dbapi_connection = connection = pool._invoke_creator(self)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 590, in connect
docker-conda-store-server-1  |     return dialect.connect(*cargs, **cparams)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 597, in connect
docker-conda-store-server-1  |     return self.dbapi.connect(*cargs, **cparams)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect
docker-conda-store-server-1  |     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
docker-conda-store-server-1  | psycopg2.OperationalError: connection to server at "postgres" (<IP_ADDRESS>), port 5432 failed: FATAL:  password authentication failed for user "postgres"
docker-conda-store-server-1  |
docker-conda-store-server-1  |
docker-conda-store-server-1  | The above exception was the direct cause of the following exception:
docker-conda-store-server-1  |
docker-conda-store-server-1  | Traceback (most recent call last):
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/bin/conda-store-server", line 33, in <module>
docker-conda-store-server-1  |     sys.exit(load_entry_point('conda-store-server', 'console_scripts', 'conda-store-server')())
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/traitlets/config/application.py", line 976, in launch_instance
docker-conda-store-server-1  |     app.start()
docker-conda-store-server-1  |   File "/opt/conda-store-server/conda_store_server/server/app.py", line 213, in start
docker-conda-store-server-1  |     self.conda_store.ensure_namespace()
docker-conda-store-server-1  |   File "/opt/conda-store-server/conda_store_server/app.py", line 342, in ensure_namespace
docker-conda-store-server-1  |     namespace = api.get_namespace(self.db, name=self.default_namespace)
docker-conda-store-server-1  |   File "/opt/conda-store-server/conda_store_server/app.py", line 279, in db
docker-conda-store-server-1  |     return self.session_factory()
docker-conda-store-server-1  |   File "/opt/conda-store-server/conda_store_server/app.py", line 269, in session_factory
docker-conda-store-server-1  |     self._session_factory = orm.new_session_factory(
docker-conda-store-server-1  |   File "/opt/conda-store-server/conda_store_server/orm.py", line 422, in new_session_factory
docker-conda-store-server-1  |     Base.metadata.create_all(engine)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/sql/schema.py", line 4888, in create_all
docker-conda-store-server-1  |     bind._run_ddl_visitor(
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3146, in _run_ddl_visitor
docker-conda-store-server-1  |     with self.begin() as conn:
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3062, in begin
docker-conda-store-server-1  |     conn = self.connect(close_with_result=close_with_result)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3234, in connect
docker-conda-store-server-1  |     return self._connection_cls(self, close_with_result=close_with_result)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 96, in __init__
docker-conda-store-server-1  |     else engine.raw_connection()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3313, in raw_connection
docker-mysql-1               |
docker-conda-store-server-1  |     return self._wrap_pool_connect(self.pool.connect, _connection)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3283, in _wrap_pool_connect
docker-mysql-1               | 2022-06-08 19:56:28+00:00 [Note] [Entrypoint]: Stopping temporary server
docker-conda-store-server-1  |     Connection._handle_dbapi_exception_noconnection(
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 2117, in _handle_dbapi_exception_noconnection
docker-conda-store-server-1  |     util.raise_(
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
docker-conda-store-server-1  |     raise exception
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 3280, in _wrap_pool_connect
docker-conda-store-server-1  |     return fn()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 310, in connect
docker-conda-store-server-1  |     return _ConnectionFairy._checkout(self)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 868, in _checkout
docker-conda-store-server-1  |     fairy = _ConnectionRecord.checkout(pool)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 476, in checkout
docker-conda-store-server-1  |     rec = pool._do_get()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/impl.py", line 256, in _do_get
docker-conda-store-server-1  |     return self._create_connection()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 256, in _create_connection
docker-conda-store-server-1  |     return _ConnectionRecord(self)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 371, in __init__
docker-conda-store-server-1  |     self.__connect()
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 665, in __connect
docker-conda-store-server-1  |     with util.safe_reraise():
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
docker-conda-store-server-1  |     compat.raise_(
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/util/compat.py", line 208, in raise_
docker-conda-store-server-1  |     raise exception
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/pool/base.py", line 661, in __connect
docker-conda-store-server-1  |     self.dbapi_connection = connection = pool._invoke_creator(self)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/create.py", line 590, in connect
docker-conda-store-server-1  |     return dialect.connect(*cargs, **cparams)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 597, in connect
docker-conda-store-server-1  |     return self.dbapi.connect(*cargs, **cparams)
docker-conda-store-server-1  |   File "/opt/conda/envs/conda-store-server/lib/python3.10/site-packages/psycopg2/__init__.py", line 122, in connect
docker-conda-store-server-1  |     conn = _connect(dsn, connection_factory=connection_factory, **kwasync)
docker-conda-store-server-1  | sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) connection to server at "postgres" (<IP_ADDRESS>), port 5432 failed: FATAL:  password authentication failed for user "postgres"
docker-conda-store-server-1  |
docker-conda-store-server-1  | (Background on this error at: https://sqlalche.me/e/14/e3q8)
docker-mysql-1               | 2022-06-08T19:56:28.882935Z 13 [System] [MY-013172] [Server] Received SHUTDOWN from user root. Shutting down mysqld (Version: 8.0.29).
docker-conda-store-server-1 exited with code 1
docker-mysql-1               | 2022-06-08T19:56:31.113404Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.29)  MySQL Community Server - GPL.
docker-mysql-1               | 2022-06-08 19:56:31+00:00 [Note] [Entrypoint]: Temporary server stopped
docker-mysql-1               |
docker-mysql-1               | 2022-06-08 19:56:31+00:00 [Note] [Entrypoint]: MySQL init process done. Ready for start up.
docker-mysql-1               |
docker-mysql-1               | 2022-06-08T19:56:32.168268Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.29) starting as process 1
docker-mysql-1               | 2022-06-08T19:56:32.185558Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started.
docker-mysql-1               | 2022-06-08T19:56:32.463988Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended.
docker-mysql-1               | 2022-06-08T19:56:32.797915Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
docker-mysql-1               | 2022-06-08T19:56:32.798064Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel.
docker-mysql-1               | 2022-06-08T19:56:32.802909Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
docker-mysql-1               | 2022-06-08T19:56:32.857573Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Bind-address: '::' port: 33060, socket: /var/run/mysqld/mysqlx.sock
docker-mysql-1               | 2022-06-08T19:56:32.857714Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.29'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.
ocker-postgres-1            | 2022-06-08 19:56:35.997 UTC [36] FATAL:  role "postgres" does not exist
docker-jupyterhub-1          | [I 2022-06-08 19:56:36.404 JupyterHub log:189] 200 GET /hub/api/ (@<IP_ADDRESS>) 1.45ms
[last two messages repeat ad infinitum]

Okay, it looks like a POSTGRES_USER environment variable needs to be set in order to create a superuser in the docker container. Making a PR now.


Windows fix for local templates.

No need to check for forward slash when checking if path is local.
Fixes #104.

@yyx990803 I think this one is ready to merged. I tested it on Windows 8 VM.


Created NewDecoder and CopyAndDecode helpers

@ahmetalpbalkan @colemickens
I factored out code previously used by ByUnmarshallingJSON and ByUnmarshallingXML into helpers. Doing so replaces the magic strings with constants and provides a CopyAndDecode helper useful in other situations.

Looks good to me, let's merge. Thanks!


Unexpected token < in JSON at position 0

It's finding the valid URLS, but then hitting an unexpected token error.
I'm in my own Zoom account, so permissions shouldn't be an issue; no passwords.
PS C:\Users\gumfa\Documents\Collab> npx zoom-rec-dl@latest
2023-04-20T17:31:28.863Z Found 3 valid URLs.
2023-04-20T17:31:28.866Z ┌ dVzedlw248B5zDYoQrsT
2023-04-20T17:31:29.161Z └ Unexpected token < in JSON at position 0
2023-04-20T17:31:29.162Z ┌ zzJlz0yqhWP2ptfGtKF5
2023-04-20T17:31:29.247Z └ Unexpected token < in JSON at position 0
2023-04-20T17:31:29.248Z ┌ yEBOOacp2-8wDVhgJgJL
2023-04-20T17:31:29.331Z └ Unexpected token < in JSON at position 0
2023-04-20T17:31:29.334Z ┌ Found 3 failed attempts.
2023-04-20T17:31:29.335Z └ Reference 2023-04-20T17-31-28.841Z/1682011889332-failed.txt

Note
Identical content was posted as a discussion. The post has been removed.

Zoom seems to be returning HTML instead of JSON.
Could you provide an example recording URL for reproduction?

https://us02web.zoom.us/rec/play/Nj2wL0PKTKNz98fVuir8xPiArDh4-Eprca58JynUMHOP4ZzjHZIv2wOcPorRyLiTHcH-q67lrQlNEQ0.rXaSDKKNDG11KWNN?autoplay=true

The provided URL requires a manual password entry. This type of link is not supported.

Please reference the requirements section of the README.

The following message will be shown in 2.1.2+. https://github.com/hyunbinseo/zoom-rec-dl/commit/c45ce3ac4e5652f5739bb76721075ac397415f9a
2023-04-26T06:48:20.625Z ┌ Nj2wL0PKTKNz98fVuir8
2023-04-26T06:48:21.297Z └ Recording does not exist. Check if the URL requires a password.

Thank you for the issue.


Update OpenTelemetry Dependencies to latest

Update OpenTelemetry Dependencies to latest

Wait for contrib 0.82.0.


apiTypes: Extract items of Stream type to StreamOptions.

As this list of streamOptions (name, description, inviteOnly,
isAnnouncementOnly, history_public_to_subscribers) will keep on
increase as we make streams more customizable. So it is good to have a
separate type for stream options.

(part of #3140)

Hmmm, I don't think I clearly understand your reasoning.
And if I am understanding it, not sure I agree with it :)
Clarify & elaborate.

@borisyankov
I guess better reasoning can be found here (by @gnprice) :).

Thanks @jainkuniya ! The quoted suggestion was:

This list of fields name, description, inviteOnly, and now isAnnouncementOnly appears in a lot of places, as your commit "createStream: Add functionality to ..." (btw "to to" -> "to") demonstrates. Also a lot of those places are positional arguments, and increasingly long lists of positional arguments make me nervous.

So the spots I was concerned about were like this one in that linked PR:
    dispatch(createNewStream(name, description, [ownEmail], isPrivate, isAnnouncementOnly));

and I'd like to group a lot of those positional arguments into an object, so they can have names.
In this spot, on the other hand, they're already part of an object and so have names attached already.


Removing 'Kaxil' from Code Owner

I won't be able to review alot of PRs given the time constraints

:'(

:) I plan to be absolutely be back in a few months with a bang. This change (I hope) will be very temporary


SW-4832 Add draft planting sites to search API

Allow clients to search for draft planting sites using whatever filter criteria
they need. Only the statically-defined fields are searchable; to fetch the opaque
data for a draft planting site, fetch it using the drafts GET endpoint.
Example payload for POST /api/v1/search to search for drafts in organization
12345:
{
    "prefix": "draftPlantingSites",
    "fields": [
        "createdBy_id",
        "description",
        "id",
        "name",
        "numPlantingSubzones",
        "numPlantingZones",
        "timeZone"
    ],
    "search": {
        "operation": "field",
        "field": "organization_id",
        "values": [12345]
    }
}

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

Current dependencies on/for this PR:

main

PR #1670

PR #1671   👈

This stack of pull requests is managed by Graphite.

Merge activity

Feb 9, 5:45 PM EST: @sgrimm started a stack merge that includes this pull request via Graphite.

@sgrimm looks like this one needs a rebase or conflict-resolution


Move parts of metricq-manager to new management baseclass

Moves parts of metricq-manager to a new management baseclass ManagementAgent for use in metricq-manager and metricq-wizard. Currently supports:

CouchDB connection
management and broadcast exchange setup
reading config from db or file
getting metrics from db

This is the sequel of https://github.com/metricq/metricq/pull/48

The wizard doesn't use this anymore. This code will not help with a potential manager refactoring. Thus, this is obsolete andwill be thrown out of the window.


Documentation Contribution

Feedback for https://armleo-openlane.readthedocs.io/en/latest/docs/source/contributing_to_docs.html#installation-of-sphinx-for-documentation
Consider one section
Contributing to OpenLane
Documentation Guide
Developer's guide
Current sections are haphazardly organized. Same type of information is strewn in different sections. see below for specifics.
Guidelines should be up, front and center.
Here's a suggested layout: (Please please organize what you plan to inform and how they will be situated logically)-> This is not a nice to have but a must and will only cause confusion when merging at the end. Users should directed logically to sections that are clearly , hierarchically titled to find the information not some kind of sphagetti flow. I cannot approve any doc changes that atleast reasonably comply with layout guidelines.
Guidelines
Writing Style and Consistency
Do's and Don'ts
Organize content from Everything under (Keep trademarks...which is not a good way to title a heading)
How to Contribute into this section (some writing sytle stuff is also here..move to Writing Style section), Header structure into suitable sub-headings under Guidelines These sections currently are all essentially guidelines and suggestions.
Installing Sphinx
Adding your Contributions
Making screenshots
...
Checking your changes
Troubleshooting
Keep titles consistent - If using progressive verbs (like Contributing..then keep that consistent.
TItle-> Contributing to OpenLane Documentation (Contributing to OpenLane Development etc)
Again- Writing style, Guidelines  ..Adhere strictly
In order to contribute to documentation, it is useful to take a look at a preview, before pushing your changes. For this purpose, you can install the sphinx and build the documentation. Should become
To contribute to documentation, check the following - something like (add a checklist)

Check Guidelines (Point to Link)
Preview your changes
Seek a Review and Approval from Project member
Commit change

Then you can view the generated html files using Firefox or other browser. Should be :
View the generated html files... (Don't use you can, just active voice...Check all your writing style to ensure this is followed and such feedback need not be repeated.
Instead of assuming directory position, explicitly state the directory to be  using cd etc. Don't encode assumptions into the script alone.
Reference link to sphinx is missing. Also reference to RST is missing.
Topics for Troubleshooting should be nicely organized. These are all randomly added now.. steps, checking installation (??),notes
List clearly the types of problems the user is likely to encounter while contributing to doc and code Not for installation. Very confusing currently.

Current sections are haphazardly organized. Same type of information is strewn in different sections. see below for specifics.

Without examples its not clear what you mean.

Guidelines should be up, front and center.

I am completely lost on the meaning of this sentence.

Here's a suggested layout: (Please please organize what you plan to inform and how they will be situated logically)-> This is not a nice to have but a must and will only cause confusion when merging at the end. Users should directed logically to sections that are clearly , hierarchically titled to find the information not some kind of sphagetti flow. I cannot approve any doc changes that atleast reasonably comply with layout guidelines.

Keep titles consistent - If using progressive verbs (like Contributing..then keep that consistent.

Need examples.

Organize content from Everything under

Sorry, I have no idea what "everything under" means in this context.

(Keep trademarks...which is not a good way to title a heading)
Renamed and restructured. Is it better?

In order to contribute to documentation, it is useful to take a look at a preview, before pushing your changes. For this purpose, you can install the sphinx and build the documentation. Should become
To contribute to documentation, check the following - something like (add a checklist)
Check Guidelines (Point to Link)
Preview your changes
Seek a Review and Approval from Project member
Commit change

Done.

Then you can view the generated html files using Firefox or other browser. Should be :
View the generated html files... (Don't use you can, just active voice...Check all your writing style to ensure this is followed and such feedback need not be repeated.

Done.

Instead of assuming directory position, explicitly state the directory to be using cd etc. Don't encode assumptions into the script alone.
Reference link to sphinx is missing. Also reference to RST is missing.

I made changes mentioning this. cd should be first command of the page, but other sections assume that each next code is running in the same terminal session as previous.

Reference link to sphinx is missing. Also reference to RST is missing.

Done.

Topics for Troubleshooting should be nicely organized. These are all randomly added now.. steps, checking installation (??),notes
List clearly the types of problems the user is likely to encounter while contributing to doc and code Not for installation. Very confusing currently.

I need examples. Currently they are organized in order of frequency.

@armleo I have provided detailed feedback - If something is not clear please reach out separately for a brief 1:1. It is difficult to cut and paste and respond to your counter questions.
Note, that without GH issues being properly understood and addressed, they will not be approved for commits, closure and will not count for project evaluations.


bump @typescript-eslint packages to v8.17.0

rebased manually via command line because we get an infinite loop of "Checking for the ability to merge automatically..."


Distinguish between cast and slot level in spell chat data

Looks great to me. Though I think it might be best to save this for next major release.


[ADD] <FRONTEND/UIcomponents>

Category Name
none
Subcategory Name
UIcomponents
Additional Context
Links for different websites which provides pre built UIcomponents.
Checklist

[X] I have checked the existing issues
[X] I have read the Contributing Guidelines
[X] I have checked that this category does not exist
[X] I have checked that this subcategory does not exist
[ ] I am willing to work on this issue (optional)
[X] I am participating in Hacktoberfest 2023

@hritikbhattacharya
I don't understand, can you elaborate what you wish to add in LinksHub?

@hritikbhattacharya
UI Libraries huh!
Looks good, what will you create? A subcategory?

@hritikbhattacharya
I have assigned you the issue. Please proceed with it, and don't hesitate to tag me @Anmol-Baranwal or @CBID2 if you require any assistance.
Make a subcategory for UI Libraries. And also add these links if possible.

yes I have done that.

Sure. Thanks for letting me know.


Digital BC broken links

Describe the bug
This page https://digital.gov.bc.ca/digital-framework is trying to link to https://www2.gov.bc.ca/gov/content/governments/services-for-government/bc-bid-resources/support-services/procurement-transformation but the original page is gone.
Additional context

Procurement changed their website

Definition of done

Find correct "Procurement strategy" link
Contact team that maintains digital-framework
Get it fixed
Follow up with Paul O (ask Julian)

The contacts for this were Strategic Procurement and Steve Chapman. Both have been contacted. Strategic Procurement confirmed the correct link and it was provided to Steve to fix.

Not doing.  No time

This is now fixed!


More portable SIMD: rem, shl, shr

Also make sure we catch the potential UB in div, rem, shl, shr.

@bors r+

:pushpin: Commit 4414d963233ab37f2c3941e7b5dde043a77d37f8 has been approved by RalfJung

:hourglass: Testing commit 4414d963233ab37f2c3941e7b5dde043a77d37f8 with merge 73cf2ba0c0c9defc8ff7c500bad6839db2864a78...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing 73cf2ba0c0c9defc8ff7c500bad6839db2864a78 to master...


Fix bug with Hidden is not shown on the header of avatar for Hidden contact

Details

Chat - "Hidden" is not shown on the header of avatar for "Hidden" contact
Fixed Issues

$ https://github.com/Expensify/App/issues/35514
PROPOSAL: https://github.com/Expensify/App/issues/35514#issuecomment-1919997266
Tests

Open 1:1 DM with Hidden user<EMAIL_ADDRESS>Click on the chat header.
Click on the avatar.
Verify that Hiden is displayed on the avatar screen

[x] Verify that no errors appear in the JS console

Offline tests

Open 1:1 DM with Hidden user<EMAIL_ADDRESS>Click on the chat header.
Click on the avatar.
Verify that Hiden is displayed on the avatar screen

QA Steps

Open 1:1 DM with Hidden user<EMAIL_ADDRESS>Click on the chat header.
Click on the avatar.
Verify that Hiden is displayed on the avatar screen

[x] Verify that no errors appear in the JS console

PR Author Checklist

[x] I linked the correct issue in the ### Fixed Issues section above
[x] I wrote clear testing steps that cover the changes made in this PR

[x] I added steps for local testing in the Tests section
[x] I added steps for the expected offline behavior in the Offline steps section
[x] I added steps for Staging and/or Production testing in the QA steps section
[x] I added steps to cover failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[x] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)
[x] I tested this PR with a High Traffic account against the staging or production API to ensure there are no regressions (e.g. long loading states that impact usability).

[x] I included screenshots or videos for tests on all platforms
[x] I ran the tests on all platforms & verified they passed on:

[x] Android: Native
[x] Android: mWeb Chrome
[x] iOS: Native
[x] iOS: mWeb Safari
[x] MacOS: Chrome / Safari
[x] MacOS: Desktop

[x] I verified there are no console errors (if there's a console error not related to the PR, report it or open an issue for it to be fixed)
[x] I followed proper code patterns (see Reviewing the code)

[x] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick)
[x] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[x] I verified that comments were added to code that is not self explanatory
[x] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[x] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method

[x] If any non-english text was added/modified, I verified the translation was requested/reviewed in #expensify-open-source and it was approved by an internal Expensify engineer. Link to Slack message:

[x] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[x] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[x] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[x] I verified the JSDocs style guidelines (in STYLE.md) were followed

[x] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[x] I followed the guidelines as stated in the Review Guidelines
[x] I tested other components that can be impacted by my changes (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar are working as expected)
[x] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[x] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[x] I verified that if a function's arguments changed that all usages have also been updated correctly
[x] If any new file was added I verified that:

[x] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[x] If a new CSS style is added I verified that:

[x] A similar style doesn't already exist
[x] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG))

[x] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[x] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[x] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[x] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[x] If the PR modifies the form input styles:

[x] I verified that all the inputs inside a form are aligned with each other.
[x] I added Design label so the design team can review the changes.

[x] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[x] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.

Screenshots/Videos

Android: Native

https://github.com/Expensify/App/assets/68128028/be8c5f88-7822-4131-b58c-53b1e03338bf

Android: mWeb Chrome

https://github.com/Expensify/App/assets/68128028/10fc7405-0254-4866-8434-f0179f175abe

iOS: Native

https://github.com/Expensify/App/assets/68128028/bc5acc4a-753a-418c-a830-673d2da93f2d

iOS: mWeb Safari

https://github.com/Expensify/App/assets/68128028/121c849a-4b77-43ec-8a00-76e32e669e34

MacOS: Chrome / Safari

https://github.com/Expensify/App/assets/68128028/3a2529f5-6c16-496d-ab07-038604faca7d

MacOS: Desktop

https://github.com/Expensify/App/assets/68128028/85ba4930-b69d-4cbf-a4ac-1a214a518047

Reviewer Checklist

[ ] I have verified the author checklist is complete (all boxes are checked off).
[ ] I verified the correct issue is linked in the ### Fixed Issues section above
[ ] I verified testing steps are clear and they cover the changes made in this PR

[ ] I verified the steps for local testing are in the Tests section
[ ] I verified the steps for Staging and/or Production testing are in the QA steps section
[ ] I verified the steps cover any possible failure scenarios (i.e. verify an input displays the correct error message if the entered data is not correct)
[ ] I turned off my network connection and tested it while offline to ensure it matches the expected behavior (i.e. verify the default avatar icon is displayed if app is offline)

[ ] I checked that screenshots or videos are included for tests on all platforms
[ ] I included screenshots or videos for tests on all platforms
[ ] I verified tests pass on all platforms & I tested again on:

[ ] Android: Native
[ ] Android: mWeb Chrome
[ ] iOS: Native
[ ] iOS: mWeb Safari
[ ] MacOS: Chrome / Safari
[ ] MacOS: Desktop

[ ] If there are any errors in the console that are unrelated to this PR, I either fixed them (preferred) or linked to where I reported them in Slack
[ ] I verified proper code patterns were followed (see Reviewing the code)

[ ] I verified that any callback methods that were added or modified are named for what the method does and never what callback they handle (i.e. toggleReport and not onIconClick).
[ ] I verified that the left part of a conditional rendering a React component is a boolean and NOT a string, e.g. myBool && <MyComponent />.
[ ] I verified that comments were added to code that is not self explanatory
[ ] I verified that any new or modified comments were clear, correct English, and explained "why" the code was doing something instead of only explaining "what" the code was doing.
[ ] I verified any copy / text shown in the product is localized by adding it to src/languages/* files and using the translation method
[ ] I verified all numbers, amounts, dates and phone numbers shown in the product are using the localization methods
[ ] I verified any copy / text that was added to the app is grammatically correct in English. It adheres to proper capitalization guidelines (note: only the first word of header/labels should be capitalized), and is approved by marketing by adding the Waiting for Copy label for a copy review on the original GH to get the correct copy.
[ ] I verified proper file naming conventions were followed for any new files or renamed files. All non-platform specific files are named after what they export and are not named "index.js". All platform-specific files are named for the platform the code supports as outlined in the README.
[ ] I verified the JSDocs style guidelines (in STYLE.md) were followed

[ ] If a new code pattern is added I verified it was agreed to be used by multiple Expensify engineers
[ ] I verified that this PR follows the guidelines as stated in the Review Guidelines
[ ] I verified other components that can be impacted by these changes have been tested, and I retested again (i.e. if the PR modifies a shared library or component like Avatar, I verified the components using Avatar have been tested & I retested again)
[ ] I verified all code is DRY (the PR doesn't include any logic written more than once, with the exception of tests)
[ ] I verified any variables that can be defined as constants (ie. in CONST.js or at the top of the file that uses the constant) are defined as such
[ ] If a new component is created I verified that:

[ ] A similar component doesn't exist in the codebase
[ ] All props are defined accurately and each prop has a /** comment above it */
[ ] The file is named correctly
[ ] The component has a clear name that is non-ambiguous and the purpose of the component can be inferred from the name alone
[ ] The only data being stored in the state is data necessary for rendering and nothing else
[ ] For Class Components, any internal methods passed to components event handlers are bound to this properly so there are no scoping issues (i.e. for onClick={this.submit} the method this.submit should be bound to this in the constructor)
[ ] Any internal methods bound to this are necessary to be bound (i.e. avoid this.submit = this.submit.bind(this); if this.submit is never passed to a component event handler like onClick)
[ ] All JSX used for rendering exists in the render method
[ ] The component has the minimum amount of code necessary for its purpose, and it is broken down into smaller components in order to separate concerns and functions

[ ] If any new file was added I verified that:

[ ] The file has a description of what it does and/or why is needed at the top of the file if the code is not self explanatory

[ ] If a new CSS style is added I verified that:

[ ] A similar style doesn't already exist
[ ] The style can't be created with an existing StyleUtils function (i.e. StyleUtils.getBackgroundAndBorderStyle(theme.componentBG)

[ ] If the PR modifies code that runs when editing or sending messages, I tested and verified there is no unexpected behavior for all supported markdown - URLs, single line code, code blocks, quotes, headings, bold, strikethrough, and italic.
[ ] If the PR modifies a generic component, I tested and verified that those changes do not break usages of that component in the rest of the App (i.e. if a shared library or component like Avatar is modified, I verified that Avatar is working as expected in all cases)
[ ] If the PR modifies a component related to any of the existing Storybook stories, I tested and verified all stories for that component are still working as expected.
[ ] If the PR modifies a component or page that can be accessed by a direct deeplink, I verified that the code functions as expected when the deeplink is used - from a logged in and logged out account.
[ ] If the PR modifies the form input styles:

[ ] I verified that all the inputs inside a form are aligned with each other.
[ ] I added Design label so the design team can review the changes.

[ ] If a new page is added, I verified it's using the ScrollView component to make it scrollable when more elements are added to the page.
[ ] If the main branch was merged into this PR after a review, I tested again and verified the outcome was still expected according to the Test steps.
[ ] I have checked off every checkbox in the PR reviewer checklist, including those that don't apply to this PR.

Screenshots/Videos

Android: Native

Android: mWeb Chrome

iOS: Native

iOS: mWeb Safari

MacOS: Chrome / Safari

MacOS: Desktop

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by https://github.com/tylerkaraszewski in version: 1.4.42-0 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅

🚀 Deployed to production by https://github.com/chiragsalian in version: 1.4.42-5 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


angular SnotifyModule Show template error

When i add SnotifyModule to the app.module.ts -> imports array [ SnotifyModule ], the page not renders and throw the error shown below to the browser console window. please guide. thanks in advance
Unhandled Promise rejection: Template parse errors:
Can't bind to 'ngIfElse' since it isn't a known property of 'div'. ("fy-prompt *ngIf="toast.config.type === state.promptType" [toast]="toast">
]*ngIf="!toast.config.icon; else elseBlock" [ngClass]="['snotify-icon', (toast.config.iconClass || 'sn"): ToastComponent@0:1167
Property binding ngIfElse not used by any directive on an embedded template. Make sure that the property name is spelled correctly and all directives are listed in the<EMAIL_ADDRESS>("snotify-prompt *ngIf="toast.config.type === state.promptType" [toast]="toast">  [ERROR ->]<div *ngIf="!toast.config.icon; else elseBlock" [ngClass]="['snotify-icon', (toast.config.iconClass |"): ToastComponent@0:1162
'ng-template' is not a known element:

If 'ng-template' is an Angular component, then verify that it is part of this module.
If 'ng-template' is a Web Component then add "CUSTOM_ELEMENTS_SCHEMA" to the<EMAIL_ADDRESS>of this component to suppress this message. ("gClass]="['snotify-icon', (toast.config.iconClass || 'snotify-icon--' + toast.config.type)]"> [ERROR ->]<ng-template #elseBlock>   "): ToastComponent@0:1312
Can't bind to 'ngIfElse' since it isn't a known property of 'div'. ("[ngStyle]="{'width': (state.progress * 100) + '%'}">
]*ngIf="!toast.config.html; else toastHTML">
{{to"): ToastComponent@0:786
Property binding ngIfElse not used by any directive on an embedded template. Make sure that the property name is spelled correctly and all directives are listed in the<EMAIL_ADDRESS>("yToast__progressBar__percentage" [ngStyle]="{'width': (state.progress * 100) + '%'}">  [ERROR ->]
 <div class="snotifyToast"): ToastComponent@0:753
'ng-template' is not a known element:

Pls, it's the vue.js repo


Getting error with Xcode 10

Describe the bug
error: module compiled with Swift 4.1.2 cannot be imported by the Swift 4.2 compiler: /Users/user/Projects/example/build/ios/Debug-iphonesimulator/BSGridCollectionViewLayout/BSGridCollectionViewLayout.swiftmodule/x86_64.swiftmodule

I'll get an Swift 4.2 version ready. In the meantime build it in Swift 4.1/4.0 compiler mode (whichever you can choose from in Xcode 10).

@mikaoj Thanks for your answer. I have selected Swift 4 in the build configuration, but it still fails with the same error.

Never mind, I guess you mean to compile your library not the project itself, as it currently is using the prebuilt binary. Will wait for you to release the 4.2 support, no rush.
Thanks.

Give 2.8.2 a try and let me know if it works out for you :)

@mikaoj Worked like a charm! Thank you very much!


Remove sshguard

sshguard is usually not needed with AppSclae since instances do not have
password enabled, and it creates issues since the various nodes need (at
this time) to be able to ssh into the head node.

OCD build https://ocd.appscale.com:8080/job/Daily Build/5145/


cmdstan package for Homebrew-science

Hi. I'm packaging cmdstan for Homebrew-science. Should the package name be cmdstan or just stan? I'm going with cmdstan, but wanted to check.
The name bin/print is too common, likely to conflict with other packages, and must be renamed. I was going to name it stanprint. Do you have a better suggestion? Would you consider also renaming it upstream?
Cheers,
Shaun

Here's the pull request for the Homebrew formula: https://github.com/Homebrew/homebrew-science/pull/2639

There's an issue with this formula with a missing header file stan/math.hpp. I can't find this file in the GitHub repository.
❯❯❯ brew test cmdstan -v
Testing homebrew/science/cmdstan
==> /usr/local/Cellar/cmdstan/2.7.0/bin/stanc --version
stanc version 2.7.0
==> /usr/local/Cellar/cmdstan/2.7.0/bin/stanc bernoulli.stan
Model name=bernoulli_model
Input file=bernoulli.stan
Output file=bernoulli_model.cpp
==> make bernoulli_model
/usr/bin/clang++ -Os -w -pipe -march=native -mmacosx-version-min=10.10 -F/usr/local/Frameworks -L/usr/local/lib -F/usr/local/Frameworks -Wl,-headerpad_max_install_names  bernoulli_model.cpp   -o bernoulli_model
In file included from bernoulli_model.cpp:3:
/usr/local/include/stan/model/model_header.hpp:4:10: fatal error: 'stan/math.hpp' file not found
#include <stan/math.hpp>
         ^
1 error generated.
make: *** [bernoulli_model] Error 1

I have no authority on this matter, but personally I prefer: cmdstan, stanprint, and stanmake. cmdstan follows the convention of installing rstan in R, pystan in Python, etc. stanprint and stanmake is what @akucukelbir uses as the names of the excellent scripts which print and make stan-related files from any directory.
On your error: To get the math header, you have to git clone recursively. Or as suggested in the wiki, call make stan-update in the cmdstan repo.

The print function is really a component of cmdstan, hence
our local naming.  I’m fine with stanprint.
Assuming that the Homebrew recipe is for the cmdstan
interface then, yes, the name should be “cmdstan”.
“stan” refers to the C++ API, not any single interface.
On Aug 10, 2015, at 8:12 AM, Shaun Jackman<EMAIL_ADDRESS>wrote:

Hi. I'm packaging cmdstan for Homebrew-science. Should the package name be cmdstan or just stan? I'm going with cmdstan, but wanted to check.
The name bin/print is too common, likely to conflict with other packages, and must be renamed. I was going to name it stanprint. Do you have a better suggestion? Would you consider also renaming it upstream?
Cheers,
Shaun
—
Reply to this email directly or view it on GitHub.

Yes to cmdstan. I'm happy changing bin/print to bin/stanprint for CmdStan (#49: we should just fix that on a separate pull request).
@sjackman, regarding the last error, we have the Stan Math Library as a submodule. Did you update that?

@dustinvtran, @akucukelbir: what is stanmake?

@syclik A version of it exists here

@syclik @dustinvtran an updated version is
stanmake () {
	inputFile=$1
	inputFileStripped=$inputFile:r
	rm -i -f $inputFileStripped.hpp
	origPWD=$PWD
	pathToProgram=$PWD/$inputFileStripped
	pathToStan=/Users/alpkucukelbir/GitHub/cmdstan/
	cd $pathToStan
	make $pathToProgram
	cd $origPWD
	rm -i -f $inputFileStripped.d
	rm -i -f $inputFileStripped.o
	rm -i -f $inputFileStripped.hpp
}

@sjackman, what's the timeline / process for getting it into Homebrew science? Is there anything else you'll need to make life easier?

I know "stanprint" is more like our current "print", but
I prefer "stansummary" if that's not too verbose, because
it's more precise.  I'd write "stanposteriorsummary" but
that's way too verbose and smashed together.
I like the idea of deprecating print if that's feasible (i.e.,
it won't conflict).   Can we do it just by copying?

Bob

On Aug 10, 2015, at 2:37 PM, Daniel Lee<EMAIL_ADDRESS>wrote:
@sjackman, what's the timeline / process for getting it into Homebrew science? Is there anything else you'll need to make life easier?
—
Reply to this email directly or view it on GitHub.

I'm using cmdstan-2.7.0.tar.gz. It includes the stan submodule, but not the math submodule. math does not appear to be a dependent submodule of cmdstan. I'd prefer not to run stan-update so that the installation does not require network access.

stansummary sounds like a good name for print.

@sjackman, what's the timeline / process for getting it into Homebrew science? Is there anything else you'll need to make life easier?

@syclik It's pretty quick. Once I've resolved the issue with the dependency on stan/math it'll be in Homebrew in a few hours, once it's been built by the @BrewTestBot.

Feel free to repackage however you want for distribution.
My understanding is that:
stan-dev/math is submodule of stan-dev/stan
stan-dev/stan is submodule of stan-dev/cmdstan

Bob

On Aug 10, 2015, at 7:36 PM, Shaun Jackman<EMAIL_ADDRESS>wrote:
I'm using cmdstan-2.7.0.tar.gz. It includes the stan submodule, but not the math submodule. math does not appear to be a dependent submodule of cmdstan. I'd prefer not to run stan-update so that the installation does not require network access.
—
Reply to this email directly or view it on GitHub.

stan-dev/math does not appear to be a git submodule of stan-dev/stan. Please do correct me if I'm wrong.

https://github.com/stan-dev/stan/blob/develop/.gitmodules

On Aug 11, 2015, at 2:48 AM, Shaun Jackman<EMAIL_ADDRESS>wrote:
stan-dev/math does not appear to be a git submodule of stan-dev/stan. Please do correct me if I'm wrong.
—
Reply to this email directly or view it on GitHub.

Ah, thanks. I was looking for it under the root directory. It's at lib/stan_math.

@bob-carpenter lib/stan_math isn't included in cmdstan-2.7.0.tar.gz. Should it be?

Not sure why this is directed at me.
From a quick look at the doc (which says 2.6.2 on the web site but
points to the 2.7.0 doc --- I created an issue to fix that), if
it's not included in the .tar.gz distribution, then we need
much better documentation to show how to include it.

Bob

On Aug 13, 2015, at 7:46 PM, Shaun Jackman<EMAIL_ADDRESS>wrote:

@bob-carpenter lib/stan_math isn't included in cmdstan-2.7.0.tar.gz. Should it be?
—
Reply to this email directly or view it on GitHub.

It is included in the tar gz file I uploaded.
https://github.com/stan-dev/cmdstan/releases/download/v2.7.0/cmdstan-2.7.0.tar.gz
GitHub doesn't include submodule inside their direct downloads.
On Aug 13, 2015 8:02 PM, "Bob Carpenter"<EMAIL_ADDRESS>wrote:

Not sure why this is directed at me.
From a quick look at the doc (which says 2.6.2 on the web site but
points to the 2.7.0 doc --- I created an issue to fix that), if
it's not included in the .tar.gz distribution, then we need
much better documentation to show how to include it.

Bob

On Aug 13, 2015, at 7:46 PM, Shaun Jackman<EMAIL_ADDRESS>wrote:

@bob-carpenter lib/stan_math isn't included in cmdstan-2.7.0.tar.gz.
Should it be?
—
Reply to this email directly or view it on GitHub.

—
Reply to this email directly or view it on GitHub
https://github.com/stan-dev/cmdstan/issues/145#issuecomment-130891829.

The Homebrew formula is working now. Here it is: https://github.com/Homebrew/homebrew-science/pull/2639
Once it builds the binary bottles, I'll merge it.

cmdstan is now in Homebrew-science. brew install homebrew/science/cmdstan

Sweet!
Shaun, sorry -- been a little busy. Was there anything else you need us
(really me) to do to make this easier going forward? If you can think of
anything, please throw up issues and I'll try to get to it.
On Fri, Aug 14, 2015 at 1:27 PM, Shaun Jackman<EMAIL_ADDRESS>wrote:

Closed #145 https://github.com/stan-dev/cmdstan/issues/145.
—
Reply to this email directly or view it on GitHub
https://github.com/stan-dev/cmdstan/issues/145#event-382803348.


Have a block-based IR for compiling, calculate max stack size

This would also be great for compiling to cranelift (especially since I modeled the block system off of their ir, a bit :upside_down_face: ), but I'm not sure if there's any way to preserve the block information all the way to the function object that you call .__jit__() on

Best I can tell, there's little performance improvement and this may be very, very slightly slower (I get stuff like norm 1.00 ± 0.01 faster than opt), but I think there's a lot more room for optimization with this compilation model than what we had before.

It'd be nice for a JIT where we just remove the interpreter loop and directly call the functions imo.

it looks like arrayvec exists in crates.io but boxvec doesn't. maybe it would be useful as an independent crate

it looks like arrayvec exists in crates.io but boxvec doesn't. maybe it would be useful as an independent crate


Question: partitioning by key (worst practises from HighLoad++)

Hey.
Not so long ago I've visited HighLoad++ where @alexey-milovidov called splitting one table to many by key as worst practise case. He announced some built-in partitioning in ClickHouse, but I haven't found anything about it in the docs (Merge engine?). My questions are:

What is this built-in partitioning and how to use it?
I may have misunderstood how to use primary-key distributing, but in my case I have CollapsingMergeTree engine table "events" with 2 billion of records. The primary key is (id, app_id). When I make query LIKE (SELECT ... WHERE app_id=X) ClickHouse runs over all 2 billions of data (not like filtering by date index). As a result a test query with multiple joins on this table works ~30 seconds. Our main case is to do analytics inside single app only, so we decided to split it into "events_<app_id>" tables and use Merge engine table "events" in order to support multi-app queries for future. The test-query on single table now works for ~3 seconds (it has not more than 300k rows).
So, what is the best practice in this case?
Thanks.

Partitioning is built into the table engines of the MergeTree family. Traditionally tables are partitioned by the month of the date column that you supply as the first parameter in the engine definition. Starting from version 54310 you can use custom partitioning keys (documentation in Russian, examples).
Using the Merge engine for partitioning is suboptimal because of the static allocation of threads to tables (some threads will finish early and the corresponding cores will be underutilized) and also because some optimizations (such as transfer of WHERE clauses to PREWHERE) are turned off.
Partitioning can improve performance because min and max values of the partition key columns are stored in each table part and these values can be used to prune table parts needed for the query. But its main usage is in facilitating data manipulation tasks (such as dropping old data). To improve performance you should choose your primary key first - if the primary key is optimally used, partitioning will not improve performance. In particular if all your queries only touch the single app_id, you should put it as the first column of the primary key.
The rule of thumb for choosing the primary key: imagine the table is sorted using the primary key value. Next imagine how the set of rows that must be read to execute the query looks like. If it consists of a few wide spans, the index will be optimally used.

Thanks, I'll have a look later.


Add view Company List command

The view company list command is added so that the user can view the list of companies in the UI. Some changes were also made to the method of switching between displaying the different lists.
Let's

add the ListCompanyCommand class
streamline the process of switching between the different lists in the UI

Codecov Report

Merging #46 (7a4d645) into master (f7019c2) will decrease coverage by 1.55%.
The diff coverage is 10.93%.

@@             Coverage Diff              @@
##             master      #46      +/-   ##
============================================
- Coverage     55.02%   53.47%   -1.56%
  Complexity      405      405
============================================
  Files            87       90       +3
  Lines          1692     1743      +51
  Branches        199      204       +5
============================================
+ Hits            931      932       +1
- Misses          720      770      +50
  Partials         41       41

Impacted Files
Coverage Δ

...rc/main/java/seedu/address/logic/LogicManager.java
68.18% <0.00%> (-3.25%)
:arrow_down:

...edu/address/logic/commands/ListCompanyCommand.java
0.00% <0.00%> (ø)

...seedu/address/logic/commands/ListEventCommand.java
0.00% <0.00%> (ø)

.../seedu/address/logic/parser/AddressBookParser.java
72.72% <0.00%> (-3.47%)
:arrow_down:

src/main/java/seedu/address/ui/CompanyCard.java
0.00% <0.00%> (ø)

...c/main/java/seedu/address/ui/CompanyListPanel.java
0.00% <0.00%> (ø)

src/main/java/seedu/address/ui/MainWindow.java
0.00% <0.00%> (ø)

...rc/main/java/seedu/address/ui/PersonListPanel.java
0.00% <0.00%> (ø)

...va/seedu/address/logic/commands/CommandResult.java
73.91% <80.00%> (-2.28%)
:arrow_down:

...java/seedu/address/logic/commands/ExitCommand.java
100.00% <100.00%> (ø)

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f7019c2...7a4d645. Read the comment docs.

LGTM!


Headless mode

This PR adds a 'headless' mode where Atmo ignores the Directive and makes each function available at its own endpoint
It also adds the internal workings for future 'remote headless' modes where Atmo uses a remote AppSource that can change over time to allow Atmo to adapt on the fly.
Resolves #33

Closing as GH workflows seem broken


Building Java Tensorflow for GPU, from Source, Does not Work as Described.

First of all, I will not address this in Stackoverflow. This question belongs here. I tried going to Stackflow but nobody responded.
I need to build tensorflow for Java from source with GPU support in Windows. How can I do this successfully? I need to build the JNI DLLs. Please advise!
I get:
bazel build --config opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni
ERROR: C:/development/projects/tensorflow/tensorflow/java/BUILD:142:1: error loading package 'tensorflow/java/src/main/native': Encountered error while reading extension file 'cuda/build_defs.bzl': no such package '@local_config_cuda//cuda': Traceback (most recent call last):
        File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 958
                _create_cuda_repository(repository_ctx)
        File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 846, in _create_cuda_repository
                _get_cuda_config(repository_ctx)
        File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 656, in _get_cuda_config
                _cudnn_install_basedir(repository_ctx)
        File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 211, in _cudnn_install_basedir
                auto_configure_fail("Cannot find cudnn install path....)
        File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 128, in auto_configure_fail
                fail("\n%sAuto-Configuration Error:%s...))

Auto-Configuration Error: Cannot find cudnn install path.
 and referenced by //tensorflow/java:libtensorflow_jni.so.
ERROR: Analysis of target '//tensorflow/java:tensorflow' failed; build aborted.
INFO: Elapsed time: 0.464s`

Please go to Stack Overflow for help and support:
http://stackoverflow.com/questions/tagged/tensorflow
If you open a GitHub issue, here is our policy:

It must be a bug or a feature request.
The form below must be filled out.

Here's why we have that policy: TensorFlow developers respond to issues. We want to focus on work that benefits the whole community, e.g., fixing bugs and adding features. Support only helps individuals. GitHub also notifies thousands of people when issues are filed. We want them to see you communicating an interesting problem, rather than being redirected to Stack Overflow.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows
TensorFlow installed from (source or binary): r1.2
TensorFlow version (use command below):r1.2
Bazel version (if compiling from source): 0.5.0
CUDA/cuDNN version: 8.0/5.1
GPU model and memory: Nvidia Quadro - K5200
Exact command to reproduce:bazel build --config opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni

You can collect some of this information using our environment capture script:
https://github.com/tensorflow/tensorflow/tree/master/tools/tf_env_collect.sh
You can obtain the TensorFlow version with
python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
Describe the problem
Describe the problem clearly here. Be sure to convey here why it's a bug in TensorFlow or a feature request.
Source code / logs
Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

I think that  error states that cudnn install path. Might be you have to configure cuda path correctly. Check installation on tensor-flow website line by line for bazel and cuda libs.

Yeah, but I do specify the CUDNN path using ./configure.

Is something hardcoded in the build script? Remember I need the dll for Windows.

Can you try to put into environment variable to have wide broader scope and try other options too.
From the error log it seems wrong config via bazel

What are the environment variables I need to set?

Has anyone been able to successfully build the Java GPU libraries for Windows?
Running:
./configure
bazel build --config opt //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni

@asimshankar, do you have any useful suggestions?

Can you try with:
bazel build --config=opt --config=cuda //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni

and let us know if that works out?

Tried your suggestion and still the same error:
`$ bazel build --config=opt --config=cuda //tensorflow/java:tensorflow //tensorflow/java:libtensorflow_jni
ERROR: no such package '@local_config_cuda//crosstool': Traceback (most recent call last):
File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 958
_create_cuda_repository(repository_ctx)
File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 846, in _create_cuda_repository
_get_cuda_config(repository_ctx)
File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 656, in _get_cuda_config
_cudnn_install_basedir(repository_ctx)
File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 211, in _cudnn_install_basedir
auto_configure_fail("Cannot find cudnn install path....)
File "C:/development/projects/tensorflow/third_party/gpus/cuda_configure.bzl", line 128, in auto_configure_fail
fail("\n%sAuto-Configuration Error:%s...))
Auto-Configuration Error: Cannot find cudnn install path.
.
INFO: Elapsed time: 0.527s
`

Unfortunately, Windows support is highly experimental. Only cmake windows builds are unit tested. As is said on the web page. So what this means is that since you didn't get a response on StackOverflow and people have told you to make sure cudnn is properly getting detected. Probably nobody else has tried this yet, so you are on a trailblazing path.

I am trying to wrap my ahead around why Windows support is highly experimental..... Isn't Windows a large installation base? Don't most of us have Nvidia cards?  Especially now that deep learning is becoming so popular, it's well worth the effort to make Windows/Java support stable.
Anyway, I will solve this myself. In the end, I may just write my own deep learning framework for Windows and Java.

I empathize that you are super frustrated, but as a developer, you must know that we don't have unlimited time to support everything ourself. You are likely one of the few people that have tried to setup Windows/Java/GPUs together. As such we really would appreciate if you solve this problem and contribute it back to the community. We hope that even going through that effort will save you time over writing your own framework. Good luck.

Thank you. I hope I find the solution, and certainly would share with the community.

Hi @nectario  @aselle  , I'm just wondering if you found a solution yet and you just forgot to mention here. Because I'm facing the same problem here.
Thanks, Harel.

Sorry, I haven't spent the time required to find a solution. :-(

Hi @nectario ,
take a look a  the script they are using to build the cpu one. It should build both java and c interface
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/windows/libtensorflow_cpu.sh
modify the script, change the line that calls configure cpu to configure gpu in the script.(the function is in another script). Later on you can call the script using bash to build.
Before you build, also change the path to the GPU libraries in these two scripts, which contains stuff called by the previous script
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/windows/bazel/common_env.sh
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/windows/bazel/bazel_test_lib.sh
Then install the bazel and all other stuff needed for bazel, including some missing command tools maybe.
Then build!

Thank you @tcmxx ! This was really helpful. Btw, I am able to successfully build using cmake, but I have no idea where it puts the Java JNI libraries. Using cmake was a lot easier and I used VS Studio to build ALL targets. Do you know where it puts the Java dlls?

Hello @nectario .
Good to hear that cmake works!
I dont know where the cmake build put the dlls but for bazel, it is under tensorflow/lib_package if everything is successful.
It is probably called libtensorflow_jni.so. You might want to look for it in the build folder...

@nectario  I have no idea. I was about to ask you how to build everything else using cmake. Never try it before...haha

Here is a screenshot of my cmake settings:
https://www.dropbox.com/s/wn3c8rdv9g0lk8n/cmake.jpg?dl=0
First I click configure, then I click generate. After generation finishes, I open the Visual Studio file it created and then build ALL.

Sorry had to update the link

Just updated with the GPU settings

@nectario I am not sure if this config has java binding. Looks like it is only for python maybe. I am not an expert of it.
If you still can not figure it out, probably it is worth trying the Bazel.

I can also send you the stuff i built. Probably you can just use it directly.

I see. Sure, thanks! Send me what you built. I can only test it when I am back from vacation (in 2 weeks). But wow, amazing that you managed to build it!

nice. i will send u the link tomorrow.
2017年8月17日 下午11:51，"nectario"<EMAIL_ADDRESS>
I see. Sure, thanks! Send me what you built. I can only test it when I am
back from vacation (in 2 weeks). But wow, amazing that you managed to build
it!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/tensorflow/tensorflow/issues/10708#issuecomment-323189738,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AGL4iOyay5-zq0lxh2Vp3zotcTVsYeoEks5sZKfQgaJpZM4N6MNq
.

Thank you so much!

@nectario https://drive.google.com/open?id=0By5_8Ik3iqjYbjBCYnJ1NmRQRUk

@tcmxx Thank you! Will give it a shot when I am back!

@tcmxx I finally got back from vacation and tried out your file! It actually worked! Nice. Thank you. Although I got the following message in the output. Seems that my Cuda capability is 3.5. Time for a new graphics card upgrade??
OUTPUT:
Hello from 1.3.0-rc2
2017-08-31 20:40:39.667550: I C:\tmp\_bazel_tcmxx\bfwkqzrk\execroot\org_tensorflow\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX
2017-08-31 20:40:40.150913: I C:\tmp\_bazel_tcmxx\bfwkqzrk\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:962] Found device 0 with properties:
name: Quadro K5200 major: 3 minor: 5 memoryClockRate(GHz): 0.771
pciBusID: 0000:83:00.0
totalMemory: 8.00GiB freeMemory: 6.71GiB
2017-08-31 20:40:40.150954: I **C:\tmp\_bazel_tcmxx\bfwkqzrk\execroot\org_tensorflow\tensorflow\core\common_runtime\gpu\gpu_device.cc:1025] Ignoring visible gpu device (device: 0, name: Quadro K5200, pci bus id: 0000:83:00.0, compute capability: 3.5) with Cuda compute capability 3.5. The minimum required Cuda capability is 3.7.**

When I try to build it now it is much better but I get the following error now:
`bazel build --config opt tensorflow/java:tensorflow tensorflow/java:libtensorflow_jni
INFO: Loading complete.  Analyzing...
WARNING: C:/development/projects/tensorflow/tensorflow/core/BUILD:1634:1: in includes attribute of cc_library rule //tensorflow/core:framework_headers_lib: '../../external/nsync/public' resolves to 'external/nsync/public' not below the relative path of its package 'tensorflow/core'. This will be an error in the future. Since this rule was created by the macro 'cc_header_only_library', the error might have been caused by the macro implementation in C:/development/projects/tensorflow/tensorflow/tensorflow.bzl:911:30.
INFO: Found 2 targets...
INFO: Building...
ERROR: missing input file '@local_jdk//:include/linux/jni_md.h'.
[6 / 449] Compiling external/protobuf_archive/src/google/protobuf/util/internal/proto_writer.cc
ERROR: C:/users/nektarios/appdata/local/temp/_bazel_nektarios/ce5f8pzu/external/protobuf_archive/BUILD:93:1: C++ compilation of rule '@protobuf_archive//:protobuf_lite' failed: msvc_cl.bat failed: error executing command external/local_config_cc/wrapper/bin/msvc_cl.bat /DOS_WINDOWS=OS_WINDOWS /DCOMPILER_MSVC /DNOGDI /DNOMINMAX /DPRAGMA_SUPPORTED /D_WIN32_WINNT=0x0600 /D_CRT_SECURE_NO_DEPRECATE /D_CRT_SECURE_NO_WARNINGS ... (remaining 33 argument(s) skipped): com.google.devtools.build.lib.shell.AbnormalTerminationException: Process terminated by signal 2.
ERROR: C:/development/projects/tensorflow/tensorflow/java/src/main/native/BUILD:67:1: //tensorflow/java/src/main/native:copy_jni_md_h: missing input file '@local_jdk//:include/linux/jni_md.h'.
ERROR: C:/development/projects/tensorflow/tensorflow/java/src/main/native/BUILD:67:1 1 input file(s) do not exist.
INFO: Elapsed time: 1.745s, Critical Path: 0.16s
`
It seems it is trying to build Linux. How do I force to build windows only?

@nectario How did you build it? Have you try to run https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/windows/libtensorflow_cpu.sh in shell?
I don remember that i had this error before..

I went to the root directory where I downloaded the tensorflow code:
(bazel is already installed)
c:\Development\Projects\tensorflow
There, I typed "configure" and specified all configuration parameters. For example, where my CUDA and CUDNN directories are etc.
Finally I typed this command:
bazel build --config opt tensorflow/java:tensorflow tensorflow/java:libtensorflow_jni
Btw, this is what Google advises in this link: https://github.com/tensorflow/tensorflow/blob/master/tensorflow/java/README.md

I did not follow this instruction.
Instead, go to https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/windows/libtensorflow_cpu.sh
I think this is the shell script that they use to build windows CPU version .
Use msys64 or any shell to run this script. It should work, or at least the errors will be easy to fix.
Then modify this above script and https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/windows/bazel/common_env.sh
https://github.com/tensorflow/tensorflow/blob/master/tensorflow/tools/ci_build/windows/bazel/bazel_test_lib.sh
to change it to GPU build
those files contains some config such as CUDA path.

Thank you. I am very close to building it now. Getting some other errors that seems compilation related...
 :-( Which Python and version are you using? Seems like some python script is failing...
ERROR: C:/tmp/_bazel_nektarios/ce5f8pzu/external/gif_archive/BUILD.bazel:8:1: C++ compilation of rule '@gif_archive//:gif' failed (Exit 1): msvc_cl.bat failed: error executing command

@nectario  my python is 3.5. Not sure what this error means. Seems like something to do with your Bazel environment? ...em...

I think the issue is from the way I installed bazel (using choco). Downloading the bazel zip I see all these missing scripts...

Good to hear! Good luck with the build then~

Thanks. Btw, what version of Bazel did you use to build?

@nectario  0.5.3 I think


long click listener

It seems like there is no way to override the default long press behavior.
Also, can't the library just use the current default theme that my app uses? Why would I have to define a theme?

@stleusc this are 2 different topics.
about the theme. no a library can'T just use the theme of another app. you will have to provide the specified attrs so the theme is working. if you set them in your theme it's fine. https://github.com/mikepenz/AboutLibraries/blob/develop/library/src/main/res/values/styles.xml#L13
Still this is not as easy as it seems :D

Sweet... thanks!


heatshrink with -w 15 and HEATSHRINK_USE_INDEX=1 loops infinitely for >= 32768 byte file

This is with revision v0.4.1-1-g7d419e1.
$ make clean
[...]
$ make OPTIMIZE="-O0"
[...]
$ ii=32768; while ((ii--)); do printf "\x90"; done > 90.bin
$ ./heatshrink -w 15 90.bin tmp
^C

Using -w 14 or lower, or building with HEATSHRINK_USE_INDEX set to zero, works around this bug.
It seems to be that in the function find_longest_match the while loop loops forever, because pos = 0, match_maxlen = 0, pospoint[match_maxlen] = pospoint[0] = 0, needlepoint[match_maxlen] = needlepoint[0] = 0x90 (presumably a byte (first one?) from the input file), hsi->index[pos] = hsi->index[0] = 0. So the first if in that while always is evaluated as true, pos gets reset to hsi->index[pos] (both are zero), and the loop continues looping.
Here's a gdb session showing the behaviour:
$ gdb --args ./heatshrink -w 15 90.bin tmp
GNU gdb (Debian 8.3.1-1) 8.3.1
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./heatshrink...
(gdb) break heatshrink_encoder.c:466
Breakpoint 1 at 0x2dee: file heatshrink_encoder.c, line 466.
(gdb) run
Starting program: /media/ssd-data/coding/proj/heatshrink/broken/heatshrink -w 15 90.bin tmp

Breakpoint 1, find_longest_match (hse=0x55555557c300, start=0, end=32768,
    maxlen=16, match_length=0x7fffffffcbdc) at heatshrink_encoder.c:466
466	    while (pos - (int16_t)start >= 0) {
(gdb) step
467	        uint8_t * const pospoint = &buf[pos];
(gdb)
468	        len = 0;
(gdb)
473	        if (pospoint[match_maxlen] != needlepoint[match_maxlen]) {
(gdb) print/x pospoint[match_maxlen]
$1 = 0x0
(gdb) print/x needlepoint[match_maxlen]
$2 = 0x90
(gdb) step
474	            pos = hsi->index[pos];
(gdb)
475	            continue;
(gdb) print/x pos
$3 = 0x0
(gdb) display/x pos
1: /x pos = 0x0
(gdb) step
466	    while (pos - (int16_t)start >= 0) {
1: /x pos = 0x0
(gdb)
467	        uint8_t * const pospoint = &buf[pos];
1: /x pos = 0x0
(gdb)
468	        len = 0;
1: /x pos = 0x0
(gdb)
473	        if (pospoint[match_maxlen] != needlepoint[match_maxlen]) {
1: /x pos = 0x0
(gdb) print/x pospoint[match_maxlen]
$4 = 0x0
(gdb) print/x needlepoint[match_maxlen]
$5 = 0x90
(gdb) step
474	            pos = hsi->index[pos];
1: /x pos = 0x0
(gdb) display/x hsi->index[pos]
2: /x hsi->index[pos] = 0x0
(gdb) step
475	            continue;
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
466	    while (pos - (int16_t)start >= 0) {
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
467	        uint8_t * const pospoint = &buf[pos];
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
468	        len = 0;
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
473	        if (pospoint[match_maxlen] != needlepoint[match_maxlen]) {
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
474	            pos = hsi->index[pos];
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
475	            continue;
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
466	    while (pos - (int16_t)start >= 0) {
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
467	        uint8_t * const pospoint = &buf[pos];
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
468	        len = 0;
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb)
473	        if (pospoint[match_maxlen] != needlepoint[match_maxlen]) {
1: /x pos = 0x0
2: /x hsi->index[pos] = 0x0
(gdb) display/x match_maxlen
3: /x match_maxlen = 0x0
(gdb)

This patch seems to fix it, but I'm not sure whether it is correct. It just fixes the infinite looping by checking that it won't loop with the same pos value.
$ git diff
diff --git a/heatshrink_encoder.c b/heatshrink_encoder.c
index edf4abe..be216ee 100644
--- a/heatshrink_encoder.c
+++ b/heatshrink_encoder.c
@@ -471,8 +471,10 @@ static uint16_t find_longest_match(heatshrink_encoder *hse, uint16_t start,
          * This is redundant with the index if match_maxlen is 0, but the
          * added branch overhead to check if it == 0 seems to be worse. */
         if (pospoint[match_maxlen] != needlepoint[match_maxlen]) {
-            pos = hsi->index[pos];
-            continue;
+            if (pos != hsi->index[pos]) {
+                pos = hsi->index[pos];
+                continue;
+            }
         }

         for (len = 1; len < maxlen; len++) {
@@ -484,6 +486,9 @@ static uint16_t find_longest_match(heatshrink_encoder *hse, uint16_t start,
             match_index = pos;
             if (len == maxlen) { break; } /* won't find better */
         }
+        if (pos == hsi->index[pos]) {
+            break;
+        }
         pos = hsi->index[pos];
     }
 #else

Curiously, -w 15 almost always ends up with negative compression for me (ie, expanding the output file to be larger than the input file). So for now I just exclude -w 15 from my scripts that determine the best compression parameters.

Thanks for your detailed bug report! This will be fixed in the next release.
I wouldn't expect -w 15 to compress effectively, except in very contrived circumstances.

Do you still intend to create a new revision of the library? I ask because I just recommended it to someone else on the freedos-devel mailing list and I stumbled across the allowed parameter ranges being larger than my script used to try. (And a -w parameter below 10 often wins out for the lDebug online help pages, as these are all below 3 KiB each.) That led me to rediscovering the -w 15 bug on large files.

Yes. It will probably be a few weeks before I get to it because I'm currently busy with moving, but I currently plan to tie up some loose ends and post another release by the end of the year.


How to change protocol id to eacape GFW detect?

Install any version softethervpn  server outside GFW,  then use SOFTETHERVPN SERVER MANAGER behind GFW   to config the searver,  GFW  will detect the vpn server and block it immediately.  So, is there anything  can do to escape GFW detect?  Thanks.

What exactly is protocol id?

@moatazelmasry2  , simplely speeking, protocol id is the flow patterns can be identified with others, for example, run one severv manager to connect any ip's 443 port, if it send back the right response, it is a se server.


Custom disconnection message

Added in custom disconnection splash screen based on @lauraselby's mod to the attendance dashboard.

Do we want to make this bit something generic?


test3

📚 $\text{Stack Overview}$
Pulls submitted in this stack:

#802 👈
main

This comment was automatically generated by st.


npm audit fix

npm audit fix as well as manually updating offending top-level packages to their latest version to remove all high severity vulnerabilities.
Note: the node-sass version (dev dependency) changes from 4.x.x to 6.x.x but otherwise no breaking changes are expected. This builds fine on my machine. Ideally, we can collect a few data points, but node-sass is pretty well supported so it should be relatively safe to merge this PR. However, it may be better to fix and merge the node-sass to sass migration first from #51.
My Machine:
macOS Big Sur
Version 11.4 (20F71)
MacBook Pro (16-inch, 2019)
Processor: 2.3 GHz 8-Core Intel Core i9

Node.js: v14.17.4
npm: v6.14.14

should be resolved in the latest version. thanks!


Automatize text generation based on results

This PR automatizes text generation based on results.
The feature is activated with the flag -output-text, then the main calls a function output_text() which performs writing of results into statistical_results.txt file.
P-values are rounded to two decimals, if p<0.01 (e.g. 0.003), it is replace by str "<0.01".
The output statistical_results.txt looks like:

Fixes: https://github.com/spine-generic/spine-generic/issues/204

@valosekj i only left minor comments-- it looks great otherwise! very useful!

@valosekj i only left minor comments-- it looks great otherwise! very useful!

Thank you! I'll implement corrections tomorrow.

@valosekj i only left minor comments-- it looks great otherwise! very useful!

I have implemented all suggestions --> should be ready for review


Usage of gradient penalty without Wasserstein Loss

Could you explain why you have used gradient penalty without Wasserstein loss?

I followed recommendations from here:
In particular,

Our fair and thorough empirical evaluation suggests that when the computational budget is limited one should consider non-saturating GAN loss and spectral normalization as default choices when applying GANs to a new dataset. Given additional computational budget, we suggest adding the gradient penalty from Gulrajani et al. (2017) and training the model until convergence.

Since computing GP is cheap in this case, I used the second suggestion.


feat: add strictMono_mersenne and corollaries

Incl. upgrade of mersenne_pos to an Iff.
Also add a positivity extension and use it in a few lemmas.

bors r+


CRUD operations for survey

[Backend] - Implement regular operations for survey(s) - create, read, update, delete

Done


Ricky

Codecov Report

Merging #12 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master      #12   +/-   ##
=======================================
  Coverage   44.12%   44.12%
=======================================
  Files          18       18
  Lines         281      281
=======================================
  Hits          124      124
  Misses        157      157

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1ad500f...96004e3. Read the comment docs.

How you know it is the last PR of the hackathon.


git: only attempt ssh-agent authentication once

This prevents infinite looping if ssh-agent authentication fails.
closes #2845

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

@bors: r+ b379a5b0a5d0a129f52f1060e0800225c1b70f7f
Thanks!

:hourglass: Testing commit b379a5b with merge 8f20561...

:sunny: Test successful - cargo-cross-linux, cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-32, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64
Approved by: alexcrichton
Pushing 8f205612833be2d99b338e679951689c0cf99322 to master...


Revert "Adds .npmignore file"

Reverts pixijs/pixi.js#2764, not necessary because whitelist already exists in package.json

cc: @englercj feel free to merge


Can not get display name in React when named export

🐛 Bug Report

To Reproduce
Steps to reproduce the behavior:

setup jest config with ts-jest

create App component and named export it as bellow.

export const App = () => <div/>;

create test file as bellow

test('<App/>', () => {
  console.log(App.name);
});

You can check App.name is undefined

Expected behavior

It should output App when I executed console.log(App.name)
envinfo
System:
    OS: Mac

Npm packages:
    jest: ^26.2.2
    ts-jest: ^26.1.4
    typescript: ^3.9.7
    babel(optional):

can you please provide your repo ?

@ahnpnl Sure.

@ahnpnl I have created repo. https://github.com/keiya01/ts-jest-issue

FYI: If I changed to compile with only babel, this issue is resolved. But I want to use ts-jest, if possible

hmm I'm not quite sure what plugins babel has, but if you write
export function App() {
  return <div/>
}

you will get value for App.name. There is a slight difference when compiling to js between function declaration vs function expression
// function declaration
exports.App = void 0;
const react_1 = __importDefault(require("react"));
function App() {
    return react_1.default.createElement("div", null);
}
exports.App = App;

// function expression
exports.App = void 0;
const react_1 = __importDefault(require("react"));
exports.App = () => {
    return react_1.default.createElement("div", null);
};

I think you can only get name if using function declaration. I suspect that Babel has a plugin to compile
exports.App = () => {
    return react_1.default.createElement("div", null);
};

to
function App() {
    return react_1.default.createElement("div", null);
}

which results in you can see value for App.name.
TypeScript compiler doesn't work the same way like Babel does, especially about transpiling ts to js. This is not ts-jest issue but just different way of compiling.

I see. Thank you for checking this issue.
Probably, I will use babel-plugin-transform-function-name.

you can also write your own TypeScript AST transformer and give it to ts-jest :), following that babel plugin does

I will give it a try :)

FYI:   This issue has also been discussed in typescript repo. https://github.com/microsoft/TypeScript/issues/6433

Ah ha nice 👍 I was also surprised that each browser has their own way of producing name so when I tested out some codes, chrome gave me different results than jest.


Add API endpoint for handling serialized data from new Soil Power Sensor

Waiting for https://github.com/jlab-sensing/soil-power-sensor-firmware/pull/24 to be merged. The jist is that there's a python package that you can add to the requirements with encode and decode functions for the measurement and response. The measurement will be decoded and the response will be encoded and sent back to the device giving it the OK to clear the most recent measurement.
Link to TestPyPI for package until it is actually published:
https://test.pypi.org/project/soil-power-sensor-protobuf/

TODO: still need to test the endpoints.
rl_endpoint: https://dirtviz.jlab.ucsc.edu/api/power-proto/
teros_endpoint: https://dirtviz.jlab.ucsc.edu/api/teros-proto/
sensor_endpoint: https://dirtviz.jlab.ucsc.edu/api/sensor/

Closing this issue as it looks like it's been implemented and tested. The leaf wetness deployment is running fairly smoothly.


Passphrase encrypted unit test

Related to
NFR As a Manufacturer,
I want to secure the recovery passphrases
So they cannot be compromised
Related to the FS The passphrase on disk will be encrypted
Action: We need a unit test to give evidence that the passphrase is encrypted

duplicated #948


Style/MethodCallWithArgsParentheses false positive with pattern matching

Expected behavior
I would expect the following code to not throw a Style/MethodCallWithArgsParentheses error because the parentheses are necessary and cannot be omitted in this situation:
case settings
  in build_minimum: true, **remainder then build(:build_minimum, settings)
  in build:, **remainder then build(:build_maximum, options)
end

Actual behavior
rubocop --debug example.rb
For /Users/bkuhlmann/Downloads: configuration from /Users/bkuhlmann/.config/rubocop/config.yml
configuration from /Users/bkuhlmann/.config/rubocop/.rubocop-https---raw-githubusercontent-com-bkuhlmann-code-quality-master-configurations-rubocop-ruby-yml
Default configuration from /Users/bkuhlmann/.gem/ruby/2.7.2/gems/rubocop-0.93.1/config/default.yml
configuration from /Users/bkuhlmann/.config/rubocop/.rubocop-https---raw-githubusercontent-com-bkuhlmann-code-quality-master-configurations-rubocop-rake-yml
configuration from /Users/bkuhlmann/.gem/ruby/2.7.2/gems/rubocop-rake-0.5.1/config/default.yml
configuration from /Users/bkuhlmann/.gem/ruby/2.7.2/gems/rubocop-rake-0.5.1/config/default.yml
configuration from /Users/bkuhlmann/.config/rubocop/.rubocop-https---raw-githubusercontent-com-bkuhlmann-code-quality-master-configurations-rubocop-performance-yml
configuration from /Users/bkuhlmann/.gem/ruby/2.7.2/gems/rubocop-performance-1.8.1/config/default.yml
configuration from /Users/bkuhlmann/.gem/ruby/2.7.2/gems/rubocop-performance-1.8.1/config/default.yml
configuration from /Users/bkuhlmann/.config/rubocop/.rubocop-https---raw-githubusercontent-com-bkuhlmann-code-quality-master-configurations-rubocop-rspec-yml
configuration from /Users/bkuhlmann/.gem/ruby/2.7.2/gems/rubocop-rspec-1.43.2/config/default.yml
configuration from /Users/bkuhlmann/.gem/ruby/2.7.2/gems/rubocop-rspec-1.43.2/config/default.yml
Inspecting 1 file
Scanning /Users/bkuhlmann/Downloads/example.rb
C

Offenses:

example.rb:17:49: C: Style/MethodCallWithArgsParentheses: Omit parentheses for method calls with arguments.
  in build_minimum: true, **remainder then build(:build_minimum, settings)
                                                ^^^^^^^^^^^^^^^^^^^^^^^^^^
example.rb:18:36: C: Style/MethodCallWithArgsParentheses: Omit parentheses for method calls with arguments.
  in build:, **remainder then build(:build_maximum, options)
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^

1 file inspected, 2 offenses detected, 2 offenses auto-correctable
Finished in 0.6471289999317378 seconds

Steps to reproduce the problem
You can reproduce this problem by adding the following configuration to your .rubocop.yml:
Style/MethodCallWithArgsParentheses:
  Enabled: true
  EnforcedStyle: omit_parentheses
  AllowParenthesesInChaining: true
  AllowParenthesesInMultilineCall: true
  IgnoreMacros: false

Next, save the following as example.rb
# frozen_string_literal: true

require "bundler/inline"

gemfile true do
  source "https://rubygems.org"
  gem "rubocop", "~> 0.93"
end

def build key, settings
  puts "#{key}, #{settings}"
end

settings = {build_minimum: true, build: "test"}

case settings
  in build_minimum: true, **remainder then build(:build_minimum, settings)
  in build:, **remainder then build(:build_maximum, settings)
end

Finally, run the following to reproduce the error:
rubocop example.rb

RuboCop version
rubocop -V
0.93.1 (using Parser <IP_ADDRESS>, rubocop-ast 0.8.0, running on ruby 2.7.2 x86_64-darwin19)

Thanks for the bug report.
This is not really related to pattern matching per say.
The cop should allow (probably) allow method calls when there's a local variable conflict (like in your example):
build = :something
build(42) # method call

You could not use parenthesis by calling self.build in your example. It would have the benefit of being clear.

:bow: @marcandre. You are correct. Using self.build works but then leads to a Style/RedundantSelf error. Renaming the private build method to something other than build ends up resolving both errors and yielding zero Rubocop violations.
What tripped me up was the original Style/MethodCallWithArgsParentheses error was misleading and I lost focus of what the root cause was.
I don't know of how to make the error message better, though? I could see Style/MethodCallWithArgsParentheses attempting to print out a better suggestion with the error but seems like too much responsibility?
I can close this issue if no one thinks there is further action here?

ℹ I'm closing this issue as I'm not sure what the best course of action is. Despite the error message being slightly confusing, upon refactoring the code as mentioned in the previous comment, the error is resolved. Probably best to see if more people stumble upon this before pursuing further.