[ISSUE #4391]optimize for printObjectProperties

Make sure set the target branch to develop
What is the purpose of the change
Optimize for MixAll#printObjectProperties
Brief change log
Determine in advance whether onlyImportantField is true and field is annotated by ImportantField.

Codecov Report

Merging #4392 (789a94d) into develop (82a48be) will decrease coverage by 0.12%.
The diff coverage is 0.00%.

@@              Coverage Diff              @@
##             develop    #4392      +/-   ##
=============================================
- Coverage      48.18%   48.05%   -0.13%
+ Complexity      5082     5065      -17
=============================================
  Files            642      642
  Lines          42770    42780      +10
  Branches        5597     5597
=============================================
- Hits           20609    20559      -50
- Misses         19655    19704      +49
- Partials        2506     2517      +11

Impacted Files
Coverage Δ

...c/main/java/org/apache/rocketmq/common/MixAll.java
44.44% <0.00%> (ø)

...va/org/apache/rocketmq/store/FlushDiskWatcher.java
81.25% <0.00%> (-9.38%)
:arrow_down:

...org/apache/rocketmq/common/stats/StatsItemSet.java
41.79% <0.00%> (-8.96%)
:arrow_down:

...in/java/org/apache/rocketmq/test/util/MQAdmin.java
38.88% <0.00%> (-5.56%)
:arrow_down:

...org/apache/rocketmq/store/ha/WaitNotifyObject.java
66.07% <0.00%> (-5.36%)
:arrow_down:

...apache/rocketmq/remoting/netty/ResponseFuture.java
85.00% <0.00%> (-5.00%)
:arrow_down:

...ketmq/common/protocol/body/ConsumerConnection.java
95.83% <0.00%> (-4.17%)
:arrow_down:

...rocketmq/remoting/netty/NettyRemotingAbstract.java
47.08% <0.00%> (-4.02%)
:arrow_down:

...ava/org/apache/rocketmq/filter/util/BitsArray.java
59.82% <0.00%> (-2.57%)
:arrow_down:

...ache/rocketmq/store/AllocateMappedFileService.java
39.02% <0.00%> (-2.44%)
:arrow_down:

... and 21 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 82a48be...789a94d. Read the comment docs.

Coverage decreased (-0.1%) to 52.125% when pulling 789a94db0f468f0aa55899239a345ed8de4a9104 on dugenkui03:optimize_for_printObjectProperties into af011b1e2d4395a5619ba9ffd27769d4d5ecdd19 on apache:develop.

Coverage decreased (-0.1%) to 52.125% when pulling 789a94db0f468f0aa55899239a345ed8de4a9104 on dugenkui03:optimize_for_printObjectProperties into af011b1e2d4395a5619ba9ffd27769d4d5ecdd19 on apache:develop.


Fixes #1993 and #2096  adds a client timeout and additional logging

This fixes #1993 and 2 parts of #2096 by adding a client timeout config option and additionally logging any status that is not http.StatusOK.
This is my first attempt at go, FYI. So I'll take any feedback you can give me (and I'm also going to have my coworkers who work in Go daily give me some feedback too)
All tests passed and so did the e2e tests
cc: @mehulkar

Excellent first-pass at Go!
It's hard to balance how we should configure this with the reality of trying to migrate our cache syncing layer to the daemon. Below you'll see the chart I made while trying to rationalize what our behavior here is.
We're in a transition between "one-off" and "long-running" configuration as some of these things will be transitioned to the daemon. Environment vars are in some kind of dead zone between "one-off" and "permanent" ("permanent" on CI, weirdly mutable on local boxes so possibly one-off), and flags really only make sense in the one-off situation.
In a future story this is a daemon configuration in the generalized case, not a per-run configuration (though no-daemon's existence necessitates per-run configuration as well). And, timeout in particular isn't really a repository-level configuration, it's a user level configuration, or even (user + repository + remote cache) level configuration. (Your distance to the cache and speed of your box isn't a shared property.)
So, to get this landed:

Two naming things: --remote-cache-timeout and TURBO_REMOTE_CACHE_TIMEOUT. (Existing vars will be renamed to that strategy in some future change.)
We need tests to ensure that we prevent setting this property via the repository config. (I don't really care if the solution for preventing it is elegant, I just want to avoid creating an API we have to deprecate.)

Flags
Env
Config File
Go: eventually read from where
Comments

--no-daemon

run.runOpts.noDaemon
Output monitoring can cause turbo to skip restoration from a cache hit. This is related but not explicitly connected.

--force
TURBO_FORCE

runcache.Opts.SkipReads → runcache.RunCache.readsDisabled
The Go names are far more expressive as to what actually happens.

--no-cache

runcache.Opts.SkipWrites → runcache.RunCache.writesDisabled
The Go names are far more expressive as to what actually happens.

--cache-dir

cache.Opts.OverrideDir → cache.fsCache.cacheDirectory
Either an absolute path, or a path relative to the repository root. Default: node_modules/.cache/turbo/

--remote-only
TURBO_REMOTE_ONLY

cache.Opts.SkipFilesystem
Disables reading and writing from the filesystem cache. Does not guarantee that the remote cache is actually enabled. Turborepo still creates cache outputs, but possibly sends them to the no-op cache if remote caching isn’t available. This exists as a workaround for unbounded growth of the cache in CI which may do their own caching from the filesystem.

cache.Opts.SkipRemote
Computed: `user.token && (repo.teamid

--cache-workers

cache.Opts.Workers
10 by default. Makes cache.Put async. When async this enables turbo to proceed to the next build task prior to moving all files into the cache.

--preflight

client.ApiClient.usePreflight
These nine items are all “how do I talk to the server?” They’re inconsistently configurable.

--token
TURBO_TOKEN VERCEL_ARTIFACTS_TOKEN
user.token
userConfig.Token() → repoConfig.GetRemoteConfig() -> client.remoteConfig -> client.ApiClient.token

TURBO_TEAMID VERCEL_ARTIFACTS_OWNER
repo.teamid
repoConfig.GetRemoteConfig() -> client.remoteConfig -> client.ApiClient.teamID
TURBO_TEAMID was incidentally created during the Viper adoption; it is entirely undocumented.

--team
TURBO_TEAM
repo.teamslug
repoConfig.GetRemoteConfig() -> client.remoteConfig -> client.ApiClient.teamSlug

--api
TURBO_API
repo.apiurl
repoConfig.GetRemoteConfig() -> client.remoteConfig -> client.ApiClient.baseUrl

--login
TURBO_LOGIN
repo.loginurl
config.RepoConfig.LoginUrl()

turbo.remoteCache.signature
cache.Opts.RemoteCacheOpts.Signature
This is 1:1 from the turbo.json parse.

turbo.remoteCache.teamId
cache.Opts.RemoteCacheOpts.TeamID
❌ Parsed but unused. Not exposed in the schema.

TURBO_REMOTE_CACHE_SIGNATURE_KEY

os.Getenv
This is only grabbed at the moment of validation.

@nathanhammond thanks!
Which file is the repository Config again? Just trying to keep all the Configs in mind. I think you're just saying don't allow the setting in the turbo.json Config file. Only allow it as a cli arg or environment variable

@tm1000
https://github.com/vercel/turbo/blob/6a0228125cafbbceed53ff17a7b25259422df568/cli/internal/config/config_file.go#L138
https://github.com/vercel/turbo/blob/6a0228125cafbbceed53ff17a7b25259422df568/cli/internal/config/config_file.go#L180-L182
So, we need to make sure that it is not read in from <repo-root>/.turbo/config.json.

@nathanhammond changes applied as requested.

@nathanhammond @chris-olszewski @mehulkar hey team!
Any update on this? I know you're all extremely busy with the amazing things at Vercel but I hate having to build my own turborepo binary and distribute it with my packages. I'd love for this to be merged/included.

@tm1000 we're tracking this to land before the 1.7 release, but don't have a strong timeline on when that will be! In other words, even if we got this PR merged today, we don't have a strong sense of when the next release would be.

@mehulkar hey that works for me! Good to know at least 1.7 :-D

@chris-olszewski completed as requested

@chris-olszewski so it's failing now due to massive changes that happened in config_file.go

@tm1000 I'll deal with the changes coming from #2382. You happen to have caught us in an awkward transition that should get a lot better once #2733 is ready and lands.

Hey @chris-olszewski.
Any updates on this? Im currently bundling my own special build of turborepo to get around client timeouts.

@NicholasLYang I'm going to move to get this merged in, but it's going to have to be ported over to Rust land.

@nathanhammond you're the best. Since it's all rust im going to close this and review the rust port. I <3 rust.


Update element lib in app and api

This is deployed and appears to be working correctly.

Codecov Report

Merging #180 into master will increase coverage by 0.10%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #180      +/-   ##
==========================================
+ Coverage   35.02%   35.13%   +0.10%
==========================================
  Files         186      186
  Lines        2995     3000       +5
  Branches      224      225       +1
==========================================
+ Hits         1049     1054       +5
  Misses       1836     1836
  Partials      110      110

Impacted Files
Coverage Δ

...ment-app/src/components/DIDDocument/DIDDocument.js
0.00% <ø> (ø)

...components/DIDDocumentEditorBar/RemoveKeyDialog.js
0.00% <ø> (ø)

packages/element-lib/src/sidetree/op/index.js
100.00% <ø> (ø)

packages/element-lib/src/__tests__/test-utils.js
94.44% <100.00%> (ø)

packages/element-lib/src/sidetree/resolve/index.js
97.02% <100.00%> (+0.09%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 29d0c46...295afcd. Read the comment docs.

Will merge after testing element-did.com one more time.


⚡ Add image upload support to JavaScript

Change List:

An image is created for each comment, and added to the comment body above the comment text.
The comment upload form will now redirect traffic through the blobstore api before forwarding it to the pre-existing comments servlet, to allow for image uploads to be handled by blobstore.

Before:

After:

depends on #39


take picture screen

Describe the bug
Hi,
im not sure if its a bug or config problem (or an low Scrren resolution issue).
When taking a picture, the pictograms are in the wrong layer (background)
Steps to reproduce the bug
set config landscape orientation resolution
resolution = (2592, 1728)
Environment

OS : Raspbian Buster
Camera: Microsoft Corp. LifeCam VX-2000 OR Canon EOS 1200D
Pibooth version: 1.2.0
Screen resolution: 1280x1024

Thanks for the feedback
It is not really a bug: pibooth is primary developped for portrait photos, and with a portrait setting you will see the pictograms.
I'll try to find a workaround to either put pictograms on foreground or hide them when the preview is in landscape.

hi, maybe the following would be a solution for landscape images.
The distance left, right and above the same size.
The pictograms for it completely downward and into the foreground
(With only one picture, you could also hide the single point)
Please don't hide the pictograms, they're so funny.

In fact the pictograms were meant to be on completely downward. To correct this I have changed the way they are loaded on the background.
However I didn't manage to find a way to have them correctly displayed on your screen and preview resolutions. There is not enough space left and I can't find a simple way to put them on top of the preview without reworking our architecture.
Here is what it looks like on a "standard" 16/10 screen with a landscape image

Hi @jo-ei ,
Does the solution described @werdeil fit your expectations?

I close this issue, this is normal that in case of wide resolution, pictograms are hidden.


2.0.0-alpha.1 Error - TypeError: Cannot read properties of undefined (reading 'map')

Hello,
I've seen the latest version of the npm package (and the default one) as of late became 2.0.0-alpha.1.
After downloading the package, I get the following error:

I know @soupette is still working on the PR regarding the compatibility with Gatsby v4 & Strapi v4. I wanted to flag out the fact that the latest alpha release has this error and ask whether we could have a timeline of when the migration might be completed - we are on the verge of using the gatsby graphql plugin, but we would love to make use of this library instead if the ETA is rather close.
Thank you!

LE: after checking out the latest readme file I have found out I need to create an API token and add it to gatsby config.
After adding it, the issue is gone and I can find my collection types.
Keep up the good work, can't wait for the release of the updated package.


Segmentation fault

Hello, thank you very much for your work.                                                                                I am having a problem as shown in the picture. My steps are 1. fine-tune the llama2 initial model with unsloth, save the weights in .safetensors format; 2. use the convert-hf.py script to convert to .m format and .t format; 3. run it on a single Raspberry Pi.                                  Hope hear from you, thanks again.

Hello @YueZhan721!
Could you run this model on a more powerful computer? How much RAM does your Raspberry Pi have?

Hello @b4rtaz  I can run it on the google colab. The Raspberry Pi I used is 5 with 8GB RAM, the .m weight file is about 2.3 GB. I'm wondering if I'm doing something wrong with the quantization or formatting process.

Llama 2 7B should be approximately 3.95GB after quantization to Q40. If it's only 2.3 GB, something might be wrong.
Could you run any model on your Raspberry Pi (for example python launch.py tinyllama_1_1b_3t_q40)?

how can I convert it from .safetensors to .m. Thanks for your reply very much.

Check convert-tokenizer-hf.py. It can convert some HF models.

Llama 2 7B should be approximately 3.95GB after quantization to Q40. If it's only 2.3 GB, something might be wrong.
Could you run any model on your Raspberry Pi (for example python launch.py tinyllama_1_1b_3t_q40)?

Yes, I can run those models(.m file) you provided. There may be something wrong when I save and convert the weight files.  Thanks sincerely! Good work, and help to me.


The story with the local mode is not over

In local mode, when the neighboring collection is hidden, an error pops up - Object 'SM_House_2_Periphery.001' can't be hidden because it is not in View Layer 'View Layer'!
https://drive.google.com/file/d/1lnyVSRabo9Bzz0JDbFnLyUKQUxW_d_J8/view?usp=sharing

Hey!
Thank you for the repro scene, I must have been missing something.
I will have a look at it this weekend and push a fix.

It should be fixed it you update to the latest version.
Thank you for reporting the issue!

The problem is gone


Possible write performance improvements

Hi,
Love this project. Haven't actually tried a data load yet but I was thinking about the write performance issues.
I see in the docs:

By default, a transaction is commited every 10 files (i.e. every 10,000,000 inserts)

This seems like a large transaction - have you tried experimenting with a much lower commit size, say 500/1000 inserts per transaction?
If that doesn't offer much improvement I did have another idea. I was wondering if splitting up the schema into several separate sqlite3 databases would help - that would enable writing to those databases parallelly.
I believe you can attach up to 10 databases with sqlite3 - so you could then query by attaching the databases together.
Once the data is loaded, a one liner could be provided for the user to execute like so to launch sqlite with everything attached in one go, e.g.:
sqlite3 database1 -cmd "$(cat attach-tables.sql)"
Where attach-tables.sql could be:
ATTACH DATABASE basic_land_property_unit AS blpu;  ATTACH DATABASE delivery_point AS dp;  ATTACH DATABASE street AS str; -- etc etc
When using sqlite3, I think querying the data should then be fairly transparent to the end user. It helps that, so long as the table names are unique in each schema, it seems you don't have to prepend the table names with the database name.
I'm not sure what performance impact this would have at query time, performing joins across the different databases and so forth.
There is the option to merge multiple databases together after you've written to them - but this would probably be expensive and have similar performance issues to writing to a single db. In fact I imagine you may well end up in a place worse than writing to a single db...

Hi @mattca thanks for your suggestions! I tried a couple of variations in transaction size but I don't think I went down to 500/1000. Worth a try indeed! Made me think that it might be a good idea to parametrise it via a flag.
In regards to the multiple databases to process in parallel, it crossed my mind but never got to the point of trying it out. I have the same feeling on the performance of merging the databases into one but I have no idea whether a join across databases will be much more expensive but might be worth a go.


Add missing arg into connectionQueueInfo.decline

https://github.com/altmp/altv-js-module/blob/rc/server/src/bindings/ConnectionInfo.cpp#L118-L122

Thanks

@C0kkie maybe it would be better to rename all the interfaces in the same style? i mean like IInterfaceName


Collections: Add metadata field to DataModel

Description
This PR adds a metadata field to collections. It uses the extras functionality; there is no schema migration.
Status

[ ] Changelog updated
[x] Ready to go

Issue #465 spawned from this.


BASE_URL / read.php

Fresh install on my unRAID Server via docker-compose.yml
When i generat a Dynamic QR Code scanning it im landing on - mydomain.com/read.php?id= -> Unique redirect identifier
The PHP-Dynamic-Qr-code is not accessible from outside atm.
The predefined qr codes that are already included work
what am I doing wrong?

I can't help you with just this little information. Just yesterday I did an installation for one person and it worked...

so can you help me with my installation via Anydesk or Teamviewer?

I can help with installation or also with custom plugin for a fee


Use active-window-border-color only when there are multiple windows in a space?

Hi!
I'd like to know if there is any way I can change the active window border color depending on the number of windows in the active space/in visible spaces.
Say I have Firefox in space 1 and four iTerm windows in space 2, I'd like to be able to differentiate active/normal iTerm windows with a yellow border on the active one, but when on space 1 I'd like Firefox to have a normal border, since it's alone in that space.
Does that make sense?
Thanks.

Hey!
Here is my solution to your problem. I execute a custom script on every window change via Signals. The script checks if there are multiple applications (aka windows) in the space and toggles the window_border property.
yabairc
# your other configs

yabai -m signal --add event=application_front_switched action="/path/to/the/custom/script"

customScript.sh
#!/usr/bin/env bash

windows="$(yabai -m query --spaces --space | jq '.windows')"

if [[ $windows == *","* ]]
then
  yabai -m config window_border on
  yabai -m config active_window_border_color "0xFFFFFF00"
else
  yabai -m config window_border off
fi

Don't forget to make your custom script executable chmod +x ./customScript.sh


[question]  Error ENOENT no such file or directory, for a few hours

Hello,
For a few hours my setup of zwavejs2mqtt installed with snap, shows the following error when trying to access it's webpage.
I tried to restart it with snap restart zwavejs2mqtt and also reboot the server but no success.

Error: ENOENT: no such file or directory, scandir '/snap/zwavejs2mqtt/244/lib/node_modules/zwavejs2mqtt/server/dist/static/css'
at Object.readdirSync (node:fs:1390:3)
at findFiles (/snap/zwavejs2mqtt/244/lib/node_modules/zwavejs2mqtt/server/lib/renderIndex.js:29:28)
at default_1 (/snap/zwavejs2mqtt/244/lib/node_modules/zwavejs2mqtt/server/lib/renderIndex.js:49:28)
at Layer.handle [as handle_request] (/snap/zwavejs2mqtt/244/lib/node_modules/zwavejs2mqtt/node_modules/express/lib/router/layer.js:95:5)
at next (/snap/zwavejs2mqtt/244/lib/node_modules/zwavejs2mqtt/node_modules/express/lib/router/route.js:137:13)
at /snap/zwavejs2mqtt/244/lib/node_modules/zwavejs2mqtt/node_modules/express-rate-limit/lib/express-rate-limit.js:170:15

How can I solve this ?
Thanks

The error is gone


chore(): upgrade to<EMAIL_ADDRESS>material@5 and<EMAIL_ADDRESS>

Description
Now that material is stable, we will update covalent to use the latest angular, material and cli.
Test Steps

[ ] npm run reinstall
[ ] See everything still working

General Tests for Every PR

[ ] ng serve --aot still works.
[ ] npm run lint passes.
[ ] npm test passes and code coverage is not lower.
[ ] npm run build still works.

Screenshots or link to StackBlitz/Plunker

Do you want to update TypeScript too?

I tried doing it, but the compiler-cli complained that it doesnt support 2.5.. :/

@emoralesb05 - fwiw, i've been running on<EMAIL_ADDRESS>on<EMAIL_ADDRESS>& the material rc for quite a while without adverse effect.

Looks like the change in Angular animations is disabling the child animation on the Demo 2:
Previously working (correct)

In this branch (incorrect)

Found another child animation issue w/ sidenavs in layouts:

Updated the components to use a grid list


move isError to jest-util

Summary
It seems nice to reuse logic here, while we wait for #2549.

Test plan
Green CI

Looks like it needs some Flow adjustments

Hmm, broke the browser test as well 🙁

Looks like we already have this on master. Correct me if I'm wrong @SimenB

Kinda. I did a version of it for async errors within jest-jasmine in #6230. Should generalize it further, but good enough 🙂


change button name

change button name from "Toggle" to "Toggle render"

Thanks. And it's better to sync this change to en docs repo if you don't mind.


wire: specify output file to generator

I would like to be able to generate wire files to something not called wire_gen.go. In my code base the convention for generated files is z_foo.go, so being able to specify the name would give me flexibility without extra commands to move the file after generation each time.
If the generator could be run with an extra argument like -o z_wire.go which would put the file in the same directory as the input wire.go that would work for me.
Other alternatives may be to use something like a --prefix "z_" argument.

Thanks @myyk for the feature request! This is definitely useful. However, -o might not be feasible if wire is generating more than one files (when there is injector in the test files). --prefix might be the way to go.

Fixed with PR #213, thanks for the contribution! Use --output_file_prefix.

can you give a full command example,

Fixed with PR #213, thanks for the contribution!


Vmapping over callables / pytest cache_clear() error

Hi @patrick-kidger,
vmap Callables
I'd like to use a vmap'd  array of Callable functions inside a dummy_diffeq that would also be vmap'd
A MWE would be something of the sort

import equinox as eqx
import jax.numpy as jnp

def vector_field(t, y, args):
    return -y

@eqx.filter_vmap(args=(0,))
def history_fn(a):
    return lambda t : jnp.array([a * t, a * t])

vmap_history_fn = history_fn(jnp.arange(10))

def dummy_diffeq(t, y0):
    return vector_field(t, y0(t), None)

@eqx.filter_vmap(args=(None, 0))
def dummy_call(t, y0):
    sol = dummy_diffeq(t, y0)
    return sol

aux = dummy_call(1.0, vmap_history_fn)

The error prompted is
 aux = dummy_call(ts, vmap_history_fn)
  File "/home/monsel/miniconda3/envs/dde/lib/python3.9/site-packages/equinox/vmap_pmap.py", line 187, in __call__
    vmapd, static = jax.vmap(
ValueError: vmap was requested to map its argument along axis 0, which implies that its rank should be at least 1, but is only 0 (its shape is ())

Unrelated pytest error
When I try to run the tests with pytest , I get errors
test/conftest.py:37: TypeError
===================================================================================================== short test summary info ======================================================================================================
ERROR test/test_adaptive_stepsize_controller.py::test_step_ts - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_adaptive_stepsize_controller.py::test_jump_ts - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_adjoint.py::test_no_adjoint - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_adjoint.py::test_backsolve - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_adjoint.py::test_adjoint_seminorm - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_adjoint.py::test_closure_errors - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_adjoint.py::test_closure_fixed - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_adjoint.py::test_implicit - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_bounded_while_loop.py::test_functional_no_vmap_no_inplace - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_bounded_while_loop.py::test_functional_no_vmap_inplace - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
ERROR test/test_bounded_while_loop.py::test_functional_vmap_no_inplace - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:

It seems that all unitary test call upon the clear_caches() when the error comes from
    @pytest.fixture(autouse=True)
    def clear_caches():
        process = psutil.Process()
        if process.memory_info().vms > 4 * 2**30:  # >4GB memory usage
            for module_name, module in sys.modules.items():
                if module_name.startswith("jax"):
                    for obj_name in dir(module):
                        obj = getattr(module, obj_name)
                        if hasattr(obj, "cache_clear"):
>                           obj.cache_clear()
E                           TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
E                               1. (self: jaxlib.xla_extension.WeakrefLRUCache) -> None
E
E                           Invoked with:

test/conftest.py:37: TypeError

If I only run test_interpolation.py for example I get :
pytest  test_interpolation.py
======================================================================================================= test session starts ========================================================================================================
platform linux -- Python 3.9.13, pytest-7.2.0, pluggy-1.0.0
rootdir: /home/monsel/Desktop/dev/diffrax
plugins: jaxtyping-0.2.7, typeguard-2.13.3
collected 1 item

test_interpolation.py E                                                                                [100%]

============================================================================================================== ERRORS ==============================================================================================================
________________________________________________________________________________________________ ERROR at setup of test_derivative _________________________________________________________________________________________________

    @pytest.fixture(autouse=True)
    def clear_caches():
        process = psutil.Process()
        if process.memory_info().vms > 4 * 2**30:  # >4GB memory usage
            for module_name, module in sys.modules.items():
                if module_name.startswith("jax"):
                    for obj_name in dir(module):
                        obj = getattr(module, obj_name)
                        if hasattr(obj, "cache_clear"):
>                           obj.cache_clear()
E                           TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
E                               1. (self: jaxlib.xla_extension.WeakrefLRUCache) -> None
E
E                           Invoked with:

conftest.py:37: TypeError
===================================================================================================== short test summary info ======================================================================================================
ERROR test_interpolation.py::test_derivative - TypeError: cache_clear(): incompatible function arguments. The following argument types are supported:
========================================================================================================= 1 error in 0.18s =========================================================================================================

Sorry about that, lmk if this is clearer now.

Regarding the vmap issue: this is expected. vmap_history_fn is a function, so the only way to pass it into a vmap'd function is as something that's broadcast. (Here you have 0 instead.)
Also, note that
@eqx.filter_vmap(args=(0,))
def history_fn(a):
    return lambda t : jnp.array([a * t, a * t])

is not allowed. The a is being captured via closure and then "smuggled out" of the vmap region.

Regrading the cache_clear error: I'm not able to reproduce this. Using JAx 0.4.2 and jaxlib 0.4.1 then I find that the tests run as normal.


How to manage apiId and apiHash in android ?

I am trying to build an android telegram client using this library. Unfortunately, I could not find any good tutorials or documentation about using this library in android. But, I've understood how the library works from the Java examples.
The problem is when implementing the library, I've to pass the api_id and api_hash to TDLib. I think these variables must be unknown to others, as this can be reused for malicious purposes by others. (Even If I obfuscate the variables, It can be decompiled)
As the documentation does not provide any best practices about storing the api_id and api_hash, How can I make them unavailable to the public and securely manage it in the android app?

On Android the values can't be accessed without root permissions if they are stored in any reasonable way. From the other side, they can be extracted from the app's memory with root permissions. So you can't do much besides storing them in the apps binary and obfuscating them in any simple way.

Alright. So just asking the worst-case scenario. What if someone decrypts the key and used it for heavy malicious activities? What will happen? Would Telegram block the apiId ?

No. Usually, only misbehaving users are blocked.

Got it chief. :+1:


Wireframes

From @ivanmauricio on November 10, 2016 12:39
We need some high fidelity wireframes.
Copied from original issue: foundersandcoders/keep-the-lights-on#10

From @iteles on November 10, 2016 15:24
Or.... low fidelity ones first to test with a couple of people/product owner (?) and then high fidelity?

keep-the-lights-on.pdf

@andrewMacmurray shouldn't be those wireframes on the wiki?


allow_unknown does not respect custom validators

The Cerberus API allows custom validation properties and custom validation types to be added by creating a class that inherits from cerberus.Validator. It also allows validation rules to be supplied for arbitrary fields, via the allow_unknown property. However, the schema for unknown properties cannot make use of custom validation properties and custom validation types.
For example:
from cerberus import Validator

class CustomValidator(Validator):

    def _validate_type_foo(self, field, value):
        if not value == "foo":
            self.error(field, "Expected a foo")

v = CustomValidator({})
v.allow_unknown = {"type": "foo"}

v.validate( { "fred": "foo", "barney": "foo" } )

I would expect the call to .validate() to approve that document, but instead
I get a traceback:
Traceback (most recent call last):
  File "test-cerberus.py", line 13, in <module>
    v.validate( { "fred": "foo", "barney": "foo" } )
  File "cerberus/cerberus.py", line 165, in validate
    return self._validate(document, schema, update=update, context=context)
  File "cerberus/cerberus.py", line 228, in _validate
    self.allow_unknown})
  File "cerberus/cerberus.py", line 118, in __init__
    self.validate_schema(schema)
  File "cerberus/cerberus.py", line 278, in validate_schema
    errors.ERROR_UNKNOWN_TYPE % value)
cerberus.cerberus.SchemaError: unrecognized data-type 'foo'

thanks should be fixed now.


Thanks!

Not an issue, I couldn't find your contact info, so just saying thank you for this awesome project! Once I figured out how to get API Gateway Proxy resource setup, she works great man.

Thanks for the feedback, Jexter. You are awesome.


sync from upstream

Pull Request type

[ ] Bugfix
[ ] Feature
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes (Please run ./gradlew generateLock saveLock to refresh dependencies)
[ ] WHOSUSING.md
[ ] Other (please describe):

Changes in this PR
Describe the new behavior from this PR, and why it's needed
Issue #
Alternatives considered
Describe alternative implementation you have considered

@v1r3n we have to merge upstream changes manually because of conflicts with DoWhileSpec.
I've created another PR for doing this, Netflix-main  -> main: https://github.com/orkes-io/conductor/pull/33


Tests: update for Blender 2.90

Made an attempt to update the tests for 2.90. Should do 1-5 of #1040. See if this looks okay.

I renamed hard-coded blender-28 names to blender.
The tests now call the blender command instead of blender-28.
At the next Blender version, it should be enough to bump the new BLENDER_MAJOR and BLENDER_MINOR variables at the top of the CI script.

Thanks for doing this @scurest!


Clocks timers v2

@hdoordt What is this PR for? Are we starting over on the clocks-timers chapter?


Which image path to be used to show static images

Hi,
I need to show the static image on home page using this template. I have tried as shown below.But it is not working. When I asked this on Stackoverflow where they're saying to use assets folder. But I cannot see such thing on this template. So can you tell me how to handle static image with this template.If you can provide an example then it's highly appreciated.
I have put the question on   Stackoverflow. But none of the solutions are working with this template.
\home\home.component.html

home.component.ts
import { Component } from '@angular/core';

@Component({
    selector: 'home',
    template: require('./home.component.html')
})
export class HomeComponent {

    public heroImageUrl ="./image/employee_management.jpg";
}

Error :
it says like this Failed to load resource: the server responded with a status of 404 (Not Found).May be a path issue.how can I give it correctly ? Image is there as shown below.

You're very close to having it working :) All you need to do is change your TypeScript code from this:
public heroImageUrl ="./image/employee_management.jpg";

... to this:
public heroImageUrl = require("./image/employee_management.jpg");

Using a require statement is how to tell Webpack to bundle the referenced asset. In the case of a jpg image, your webpack.config.js is already configured to use url-loader for jpg images, which means the require statement will return a URL matching the bundled image.
Note for anyone who just wants to put images in html templates: If you just want to reference an image from inside an html file, and you don't need to get its URL as a variable in your TypeScript code, then you don't have to use require at all. .html files are already configured to use html-loader, which automatically replaces fixed src attribute values on image tags with the bundled image URL. For example, you can write the following in one of your Angular component templates:

... and that will work just the same. Webpack will bundle the referenced image and change the src attribute to be the URL of the bundled file. You only need to use require if you're trying to obtain the URL inside your TypeScript code.

When I asked this on Stackoverflow where they're saying to use assets folder. But I cannot see such thing on this template.

You can totally use a static assets folder if you want. Just create some directory under wwwroot and put images in there. Then you can reference images from there in the usual way - an image on disk at wwwroot/myimages/someimage.jpg will be reachable at the url /myimages/someimage.jpg. If you do this, you are not using Webpack to bundle those images (files under wwwroot just get deployed independently of Webpack) so don't try to use require to resolve paths to URLs for those.

BTW I've also posted an answer on StackOverflow. If you can mark it as correct, this will avoid confusion for others.

Aha.. Awesome. Thanks a lot  :smile:

@SteveSandersonMS If I do the same (latest ASP.NET Core 2.0 preview 2) I get this error with the ASP.NET Core default error page:
You may need an appropriate loader to handle this file type.
even though I have this in my webpack.config.js on the clientBundleConfig:
{ test: /\.(png|jpg|jpeg|gif|svg)$/, use: 'url-loader?limit=25000' }
My guess would be that it is because this is not on the serverBundleConfig. So I added it. Now the error is gone, but in the output DOM the only thing visible is: . This obviously shows that the import/require is now working but not being "exposed" to the browser.


Problem with glInvalidateFramebuffer() in WebGL2 on Chrome with NVIDIA GPUs.

Just tracking this issue here too, it doesn't seem to be a bug in sokol_gfx.h.
Either the problem can be fixed in Chrome, or sokol_gfx.h shouldn't call glInvalidateFramebuffer().
https://bugs.chromium.org/p/chromium/issues/detail?id=1453425

Doh, simple bug on my side, see comment here:
https://bugs.chromium.org/p/chromium/issues/detail?id=1453425#c10


Correct some Chinese (including Simplified Chinese and Traditional Chinese) errors, and modify some grammar and details.

Thank you for the updated language! Merged.


Update queries ansible aws #2103

Closes #2103
Proposed Changes
Changing all functions to import
I submit this contribution under Apache-2.0 license.

Great work on this PR, this was a needed refactor for Ansible queries. I've also associated this PR with issue #1797


break up bookinfo sample into tasks

Here is my first pass at breaking out the bookinfo demo steps into a set of tasks.
PTAL

I will merge this now, since it is just a WIP and we can continue to improve it in others PRs.


CI: Cancel GH Actions jobs after 90 min max

In hope to prevent future stale CI jobs from blocking the queue again
Doesn't affect all jobs, only those where we've seen stalling

I've merged sth similar for MNE-BIDS and it seems to be working as intended!
PR: https://github.com/mne-tools/mne-bids/pull/760
In action:
https://github.com/mne-tools/mne-bids/pull/759/checks?check_run_id=2220090841
(the run got stalled there)

Thanks @hoechenberger !


Recommended development process for building docker images?

In investigating #1786, @shotgunosine and I tried to build new Docker images of fMRIprep as recommended in the development guidelines. Unfortunately, running the built images fails with the error:
Traceback (most recent call last):
  File "/usr/local/miniconda/bin/fmriprep", line 10, in <module>
    sys.exit(main())
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 311, in main
    opts = get_parser().parse_args()
  File "/usr/local/miniconda/lib/python3.7/site-packages/fmriprep/cli/run.py", line 49, in get_parser
    currentv = Version(__version__)
  File "/usr/local/miniconda/lib/python3.7/site-packages/packaging/version.py", line 221, in __init__
    raise InvalidVersion("Invalid version: '{0}'".format(version))
packaging.version.InvalidVersion: Invalid version: ''

Is there a new, recommended method for building images ? Here, we simply modified setup.cfg and created the image with
docker build -t fmriprep-t2s .

Ah, yeah, those do not reflect the current process:
https://github.com/poldracklab/fmriprep/blob/a6029eda7a418cea2d9f7326e6d455afa2c5acb4/.circleci/config.yml#L61-L67
And the critical one is --build-arg VERSION, and THISVERSION is just the output of python get_version.py. So:
docker build -t fmriprep-t2s --build-arg VERSION=$( python get_version.py ) .

I'll update the docs. Thanks for pointing this out.

@emdupre If you care to review the updated docs: #1797.

Closed with #1797


POJO AvroSchema always allowNull

Describe the bug
Currently AvroSchema makes all fields allowNull, which makes the generated schema doesn't respect what POJO defines.
Expected behavior
AvroSchema should respect to what POJO defines
Additional context
N/A

@codelipenghui 's team is working on this issue.

@sijie I don't quite follow.  If a class has a field "Integer" than the field shoulda allow vs if the field is "int" then it shouldn't:
 public static class Foo {
        int field1;
        Integer field2;
    }

        log.info("{}", new String(AvroSchema.of(Foo.class).getSchemaInfo().getSchema()));

produces:
{"type":"record","name":"Foo","namespace":"org.company.Test4$","fields":[{"name":"field1","type":"int"},{"name":"field2","type":["null","int"],"default":null}]}

which is correct.  Am I missing something?

@jerrypeng
the main problem of this is : if we are serializing POJO into avro formats, it should be respecting to how AVRO generates schema for POJOs. Otherwise, if you are migrating other AVRO data from other data sources to Pulsar, or from Pulsar to other data destinations, it will raise schema incompatibility issue. Because Pulsar schema generated schema always allow null, while the schema generated by other systems don't allow null, even they are generated from same POJO.
AVRO is a pretty standard data format. If pulsar is using AVRO schema, we should respect to its rules, otherwise it is impossible for pulsar to be used for exchanging AVRO data.

@sijie I don't understand why you say we are not respecting the rules of AVRO, we are using the official java AVRO library to schema generation.  And if another system is using AVRO java library to generate POJOs (java objects) then they will get the same result.  Are you talking about cross language support? The only flag that could be settable is the allow null flag that gets set in Pulsar's Avro schema.  We can expose that flag to users.

@jerrypeng usually we create POJO by defining avro schema file and create it by using avro maven plugin. Once we provide the generated POJO to pulsar producer the schema registered in registry not identical to schema we have written. This is because of allowNull as pointed out by sijieg . This came as an surprise . When we tried consuming with Python with the same schema it failed because of schema incompatibility. If people want null to all the fields then they can add it at the avro schema.

Exposing the flag may also work with default as allowNull to false .

@skyrocknroll @sijie I don't think the current API is the problem, I think we just need to separate API to specify a custom schema.  We currently only allow users to derive a schema from a Java class using the Avro library.  This generates an Avro schema that matches the semantics of java i.e. classes can be null and primitives are not.  However, if I have my own custom avro schema, I can arbitrarily set one class to be nullable while the another class is not.  There is not way for Avro to know that information from just a java class definition.  Even by allowing users to set the flag "allowNull" will not cover all cases as user can arbitrarily create avro schemas will null or not null fields.  I think we can do the following:

Use  Avro java annotation to massage your java class to have the schema you desire
Have an API for users to specify the schema they want.

Irrespective of the allowNull=true/false question, I think:

This seems to be a problem only with Avro generated code, in which there was a schema defined and we're extracting the wrong schema from the generated POJO
This shouldn't be an issue when one defines an ad-hoc POJO (without starting from Avro schema), because annotations can be used to request nullable/non-nullable.

The easiest solution would be to detect that a POJO is generated by Avro and use the getClassSchema() instead of doing the reflection based approach.

I don't understand why you say we are not respecting the rules of AVRO
The only flag that could be settable is the allow null flag that gets set in Pulsar's Avro schema. We can expose that flag to users.

@jerrypeng
That is the problem I am reporting this in the issue. Because Pulsar Avro Schema always set AllowNull and pulsar's users don't have the ability to bypass the AllowNull. My point here is Pulsar AVRO should not try to attach AllowNull.

If people want null to all the fields then they can add it at the avro schema.

@skyrocknroll exactly.

I don't think the current API is the problem, I think we just need to separate API to specify a custom schema.

Jerry, that's a different feature.
The point here for a given POJO, Pulsar should maintain a basic contract: the schema generated by Pulsar AVRO should be same/compatible with what other tools generated. Pulsar AVRO should not automatically attach AllowNull for all fields, otherwise we shouldn't call it AVRO schema, we should be calling it AlwaysAllowNullAvroSchema.

This seems to be a problem only with Avro generated code, in which there was a schema defined and we're extracting the wrong schema from the generated POJO
This shouldn't be an issue when one defines an ad-hoc POJO (without starting from Avro schema), because annotations can be used to request nullable/non-nullable.

@merlimat : that's not the only case. People can use ReflectData to generate a schema and write events in a different system (let's call it source). when people migrate data from source to pulsar, people use same POJO, these two schemas are not compatible, which will result in events are not able to be written to pulsar.

The easiest solution would be to detect that a POJO is generated by Avro and use the getClassSchema() instead of doing the reflection based approach.

That's a very limited solution to cover only one use case.

In summary, the questions is about "shall Pulsar automatically attach AllowNull to POJO generated schema". Attaching AllowNull changes the default behavior of how AVRO generates POJO and makes Pulsar inconsistent with how other systems use AVRO.
If the team decides that AllowNull should be the default behavior in AvroSchema, it should be well documented and highlighted to all avro schema users. and a separate AvroSchema or a flag to disable AllowNull should also be provided for users to not use AllowNull.
I don't think detecting if it is an Avro generated POJO is the right approach. We should just remove (or provide the ability to) AllowNull when constructing AvroSchema.

that's not the only case. People can use ReflectData to generate a schema and write events in a different system (let's call it source). when people migrate data from source to pulsar, people use same POJO, these two schemas are not compatible, which will result in events are not able to be written to pulsar.

Given a POJO generated by Avro, there is no way to determine whether this POJO was generated with a schema that allowed or not for null types.

That's a very limited solution to cover only one use case.

It's a 100% correct solution for that case. I don't see what's limited about that.

Though, I too think we should remove the AllowNull from the generator, but that's separate from the point that that alone won't guarantee we can generate the correct schema starting from the generated POJO.

The point here for a given POJO, Pulsar should maintain a basic contract: the schema generated by > Pulsar AVRO should be same/compatible with what other tools generated. Pulsar AVRO should not > > automatically attach AllowNull for all fields, otherwise we shouldn't call it AVRO schema, we should > > be calling it AlwaysAllowNullAvroSchema.

@sijie the contract you are creating when generating an avro schema using the ReflectData api is with the java class itself not some other tool or system.  If you want to provide a custom schema, we should have the way to do so.  These issues are connected.
Like I mentioned before, a user can create a Avro schema file with arbitrary fields set to null.  There is not way to get such information from a java class along.  Thus, the api currently does not suffice for general use cases when users are bring in generic avro schemas irregardless of whether allowNull is set by default.
To summarize, we can NOT set allowNull by default but it hardly solves the underlying issue.

Given a POJO generated by Avro, there is no way to determine whether this POJO was generated with a schema that allowed or not for null types.
but that's separate from the point that that alone won't guarantee we can generate the correct schema starting from the generated POJO.

@merlimat
The issue I am creating here is for AllowNull. We found AllowNull is a problem from 2 use cases: 1) the one reported by @skyrocknroll  2) the other use case that @codelipenghui hit.
The whole picture of @skyrocknroll s problem is  avro file => avro generated pojo. a schema generated by avro file is not compatible by scheme generated by pulsar.AvroSchema(generated pojo). One of the problems is AllowNullcompletely changes the schema definitions. Whether removingAllowNullcan address this problem or not is a separate issue to be address. Although I would expect Avro can handle this well. We shouldn't couple the discussion of this issue with a broad issue introduced byAllowNull`.

It's a 100% correct solution for that case. I don't see what's limited about that.

@merlimat
Generated POJO is the use case reported and discussed at slack channel. If you just handle generated POJO by using getClassSchema, you are not covering many other data sources which generate AVRO schema using ReflectData.

the contract you are creating when generating an avro schema using the ReflectData api is with the java class itself not some other tool or system.

@jerrypeng
I am not creating any contract. They are from real use cases. Also the whole discussion is around POJO only, it is not even a cross-language issue or any user customized schema issue. (it was found when being used in cross-language)

if a user uses an AVRO schema file to generate a schema using avro tools (that's call this schema A), and generate a POJO class. then the user use the POJO class and use pulsar avro schema to generate another schema B. Ideally A and B should be compatible.

schema A => generated pojo => (pulsar AvroSchema) => schema B

                       ||
                       \/

schema A => generated pojo => ReflectData.AllowNull.parse => schema B

AllowNull is the problem to prevent them being compatible. I don't know if removing AllowNull can fully address this problem or not. That's a specific issue to address for Avro generated POJO. I hope Avro provides the right tools to convert back-and-forth between schema and pojos, otherwise IMO it is a problem of Avro.
If we removed AllowNull, the flow will be changed to following. The scope of the problem is different - whether ReflectData is the right tool to handle Avro generated POJO or does Avro even provide tools to guarantee such conversations.
schema A => generated pojo => ReflectData.parse => schema B

Image I have a data system A (e.g. Spark or Flink) and Pulsar. I have a POJO class (e.g. UserProfile) defined across the whole organization. The schema generated in different systems are completely not compatible even they are using same POJO. When data is flowing between pulsar and other systems, the data might not be processed properly due to incompatible schema. Pulsar is the message bus for exchanging data between other system. If it produces an incompatible schema than other systems, IMO that's a very serious bug.

#3752

@sijie that not going to work.  If you take look at the java code generated by avro tools with a given schema file its going to be not what you expect.  And from that generated class there is no way for reflect data avro library to generate the same schema unless we do the trick @merlimat mentioned

@jerrypeng

And from that generated class there is no way for reflect data avro library to generate the same schema unless we do the trick @merlimat mentioned

we can do the trick for what @merlimat mentioned for addressing the specific problem of avro generated code. but that's the separate problem of AllowNull.

@sijie actually this does work:
schema A => generated pojo => ReflectData.parse => schema B
Schema A == . Schema B

@jerrypeng

Using the reflect data parser in avro will produce the same schema.

Glad we are on the same page now.

I agree we should remove AllowNull to be set by default, but we need a migration plan

I don't think we need a migration plan. We just need a flag to tell whether AllowNull is used or not. Explained in #3752 . Migration is a nightmare. I don't want to touch it.

We are using avro because avro being an standard and guaranteed to work across different part of the  whole data pipeline. Any system we use in the pipeline we will just check does it adhere to avro standards. To add more clarity , We force every engineer to first create an avro schema and get it reviewed by data platform team. We generate pojo for  java , Python and go struts. Any system violating the avro standards will breaks the system.

@skyrocknroll I guess, for Python it might make sense to also allow to pass the JSON avro definition

@merlimat Yes, that would be awesome!

@sijie @skyrocknroll I just experimented a little.
Given the schema that @skyrocknroll posted on the pulsar chat as an example:
{
  "namespace": "example.avro",
  "type": "record",
  "name": "User",
  "fields": [
    {
      "name": "name",
      "type": "string"
    },
    {
      "name": "favorite_number",
      "type": [
        "int",
        "null"
      ]
    },
    {
      "name": "favorite_color",
      "type": "string"
    },
    {
      "name": "age",
      "type": "int",
      "default": 19
    }
  ]
}

I did some tests using the generated java class with the ReflectData schema generator:
log.info("{}", new String(AvroSchema.of(User.class).getSchemaInfo().getSchema()));

log.info("{}", ReflectData.get().getSchema(User.class));

log.info("{}", ReflectData.AllowNull.get().getSchema(User.class));

They all produced the same result:
21:07:27.741 [main:org.company.Test4@44] INFO  org.company.Test4 - {"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}

21:07:27.746 [main:org.company.Test4@46] INFO  org.company.Test4 - {"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}

21:07:27.746 [main:org.company.Test4@48] INFO  org.company.Test4 - {"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}

regardless of  whether AllowNull is set or not.  So my question still holds what exactly does removing AllowNull change?

Here is actually the line of code that actually just gets the "SCHEMA" field regardless of the java class:
https://github.com/apache/avro/blob/release-1.8.2/lang/java/avro/src/main/java/org/apache/avro/specific/SpecificData.java#L277

@jerrypeng
I am surprised to see {"name":"favorite_color","type":"string"}is not a union type with null after using AllowNull.
so my question is how do you do the test? Is the User any kind of POJO class, or avro generated POJO class?

@sijie The User class is an Avro generated class based on the schema file.
/**
 * Autogenerated by Avro
 *
 * DO NOT EDIT DIRECTLY
 */
package example.avro;

import org.apache.avro.specific.SpecificData;
import org.apache.avro.message.BinaryMessageEncoder;
import org.apache.avro.message.BinaryMessageDecoder;
import org.apache.avro.message.SchemaStore;

@SuppressWarnings("all")
@org.apache.avro.specific.AvroGenerated
public class User extends org.apache.avro.specific.SpecificRecordBase implements org.apache.avro.specific.SpecificRecord {
  private static final long serialVersionUID = -7831158737375082161L;
  public static final org.apache.avro.Schema SCHEMA$ = new org.apache.avro.Schema.Parser().parse("{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"example.avro\",\"fields\":[{\"name\":\"name\",\"type\":\"string\"},{\"name\":\"favorite_number\",\"type\":[\"int\",\"null\"]},{\"name\":\"favorite_color\",\"type\":\"string\"},{\"name\":\"age\",\"type\":\"int\",\"default\":19}]}");
  public static org.apache.avro.Schema getClassSchema() { return SCHEMA$; }

  private static SpecificData MODEL$ = new SpecificData();

  private static final BinaryMessageEncoder<User> ENCODER =
      new BinaryMessageEncoder<User>(MODEL$, SCHEMA$);

  private static final BinaryMessageDecoder<User> DECODER =
      new BinaryMessageDecoder<User>(MODEL$, SCHEMA$);

  /**
   * Return the BinaryMessageDecoder instance used by this class.
   */
  public static BinaryMessageDecoder<User> getDecoder() {
    return DECODER;
  }

  /**
   * Create a new BinaryMessageDecoder instance for this class that uses the specified {@link SchemaStore}.
   * @param resolver a {@link SchemaStore} used to find schemas by fingerprint
   */
  public static BinaryMessageDecoder<User> createDecoder(SchemaStore resolver) {
    return new BinaryMessageDecoder<User>(MODEL$, SCHEMA$, resolver);
  }

  /** Serializes this User to a ByteBuffer. */
  public java.nio.ByteBuffer toByteBuffer() throws java.io.IOException {
    return ENCODER.encode(this);
  }

  /** Deserializes a User from a ByteBuffer. */
  public static User fromByteBuffer(
      java.nio.ByteBuffer b) throws java.io.IOException {
    return DECODER.decode(b);
  }

  @Deprecated public java.lang.CharSequence name;
  @Deprecated public java.lang.Integer favorite_number;
  @Deprecated public java.lang.CharSequence favorite_color;
  @Deprecated public int age;

  /**
   * Default constructor.  Note that this does not initialize fields
   * to their default values from the schema.  If that is desired then
   * one should use newBuilder().
   */
  public User() {}

  /**
   * All-args constructor.
   * @param name The new value for name
   * @param favorite_number The new value for favorite_number
   * @param favorite_color The new value for favorite_color
   * @param age The new value for age
   */
  public User(java.lang.CharSequence name, java.lang.Integer favorite_number, java.lang.CharSequence favorite_color, java.lang.Integer age) {
    this.name = name;
    this.favorite_number = favorite_number;
    this.favorite_color = favorite_color;
    this.age = age;
  }

  public org.apache.avro.Schema getSchema() { return SCHEMA$; }
  // Used by DatumWriter.  Applications should not call.
  public java.lang.Object get(int field$) {
    switch (field$) {
    case 0: return name;
    case 1: return favorite_number;
    case 2: return favorite_color;
    case 3: return age;
    default: throw new org.apache.avro.AvroRuntimeException("Bad index");
    }
  }

  // Used by DatumReader.  Applications should not call.
  @SuppressWarnings(value="unchecked")
  public void put(int field$, java.lang.Object value$) {
    switch (field$) {
    case 0: name = (java.lang.CharSequence)value$; break;
    case 1: favorite_number = (java.lang.Integer)value$; break;
    case 2: favorite_color = (java.lang.CharSequence)value$; break;
    case 3: age = (java.lang.Integer)value$; break;
    default: throw new org.apache.avro.AvroRuntimeException("Bad index");
    }
  }

  /**
   * Gets the value of the 'name' field.
   * @return The value of the 'name' field.
   */
  public java.lang.CharSequence getName() {
    return name;
  }

  /**
   * Sets the value of the 'name' field.
   * @param value the value to set.
   */
  public void setName(java.lang.CharSequence value) {
    this.name = value;
  }

  /**
   * Gets the value of the 'favorite_number' field.
   * @return The value of the 'favorite_number' field.
   */
  public java.lang.Integer getFavoriteNumber() {
    return favorite_number;
  }

  /**
   * Sets the value of the 'favorite_number' field.
   * @param value the value to set.
   */
  public void setFavoriteNumber(java.lang.Integer value) {
    this.favorite_number = value;
  }

  /**
   * Gets the value of the 'favorite_color' field.
   * @return The value of the 'favorite_color' field.
   */
  public java.lang.CharSequence getFavoriteColor() {
    return favorite_color;
  }

  /**
   * Sets the value of the 'favorite_color' field.
   * @param value the value to set.
   */
  public void setFavoriteColor(java.lang.CharSequence value) {
    this.favorite_color = value;
  }

  /**
   * Gets the value of the 'age' field.
   * @return The value of the 'age' field.
   */
  public java.lang.Integer getAge() {
    return age;
  }

  /**
   * Sets the value of the 'age' field.
   * @param value the value to set.
   */
  public void setAge(java.lang.Integer value) {
    this.age = value;
  }

  /**
   * Creates a new User RecordBuilder.
   * @return A new User RecordBuilder
   */
  public static example.avro.User.Builder newBuilder() {
    return new example.avro.User.Builder();
  }

  /**
   * Creates a new User RecordBuilder by copying an existing Builder.
   * @param other The existing builder to copy.
   * @return A new User RecordBuilder
   */
  public static example.avro.User.Builder newBuilder(example.avro.User.Builder other) {
    return new example.avro.User.Builder(other);
  }

  /**
   * Creates a new User RecordBuilder by copying an existing User instance.
   * @param other The existing instance to copy.
   * @return A new User RecordBuilder
   */
  public static example.avro.User.Builder newBuilder(example.avro.User other) {
    return new example.avro.User.Builder(other);
  }

  /**
   * RecordBuilder for User instances.
   */
  public static class Builder extends org.apache.avro.specific.SpecificRecordBuilderBase<User>
    implements org.apache.avro.data.RecordBuilder<User> {

    private java.lang.CharSequence name;
    private java.lang.Integer favorite_number;
    private java.lang.CharSequence favorite_color;
    private int age;

    /** Creates a new Builder */
    private Builder() {
      super(SCHEMA$);
    }

    /**
     * Creates a Builder by copying an existing Builder.
     * @param other The existing Builder to copy.
     */
    private Builder(example.avro.User.Builder other) {
      super(other);
      if (isValidValue(fields()[0], other.name)) {
        this.name = data().deepCopy(fields()[0].schema(), other.name);
        fieldSetFlags()[0] = true;
      }
      if (isValidValue(fields()[1], other.favorite_number)) {
        this.favorite_number = data().deepCopy(fields()[1].schema(), other.favorite_number);
        fieldSetFlags()[1] = true;
      }
      if (isValidValue(fields()[2], other.favorite_color)) {
        this.favorite_color = data().deepCopy(fields()[2].schema(), other.favorite_color);
        fieldSetFlags()[2] = true;
      }
      if (isValidValue(fields()[3], other.age)) {
        this.age = data().deepCopy(fields()[3].schema(), other.age);
        fieldSetFlags()[3] = true;
      }
    }

    /**
     * Creates a Builder by copying an existing User instance
     * @param other The existing instance to copy.
     */
    private Builder(example.avro.User other) {
            super(SCHEMA$);
      if (isValidValue(fields()[0], other.name)) {
        this.name = data().deepCopy(fields()[0].schema(), other.name);
        fieldSetFlags()[0] = true;
      }
      if (isValidValue(fields()[1], other.favorite_number)) {
        this.favorite_number = data().deepCopy(fields()[1].schema(), other.favorite_number);
        fieldSetFlags()[1] = true;
      }
      if (isValidValue(fields()[2], other.favorite_color)) {
        this.favorite_color = data().deepCopy(fields()[2].schema(), other.favorite_color);
        fieldSetFlags()[2] = true;
      }
      if (isValidValue(fields()[3], other.age)) {
        this.age = data().deepCopy(fields()[3].schema(), other.age);
        fieldSetFlags()[3] = true;
      }
    }

    /**
      * Gets the value of the 'name' field.
      * @return The value.
      */
    public java.lang.CharSequence getName() {
      return name;
    }

    /**
      * Sets the value of the 'name' field.
      * @param value The value of 'name'.
      * @return This builder.
      */
    public example.avro.User.Builder setName(java.lang.CharSequence value) {
      validate(fields()[0], value);
      this.name = value;
      fieldSetFlags()[0] = true;
      return this;
    }

    /**
      * Checks whether the 'name' field has been set.
      * @return True if the 'name' field has been set, false otherwise.
      */
    public boolean hasName() {
      return fieldSetFlags()[0];
    }

    /**
      * Clears the value of the 'name' field.
      * @return This builder.
      */
    public example.avro.User.Builder clearName() {
      name = null;
      fieldSetFlags()[0] = false;
      return this;
    }

    /**
      * Gets the value of the 'favorite_number' field.
      * @return The value.
      */
    public java.lang.Integer getFavoriteNumber() {
      return favorite_number;
    }

    /**
      * Sets the value of the 'favorite_number' field.
      * @param value The value of 'favorite_number'.
      * @return This builder.
      */
    public example.avro.User.Builder setFavoriteNumber(java.lang.Integer value) {
      validate(fields()[1], value);
      this.favorite_number = value;
      fieldSetFlags()[1] = true;
      return this;
    }

    /**
      * Checks whether the 'favorite_number' field has been set.
      * @return True if the 'favorite_number' field has been set, false otherwise.
      */
    public boolean hasFavoriteNumber() {
      return fieldSetFlags()[1];
    }

    /**
      * Clears the value of the 'favorite_number' field.
      * @return This builder.
      */
    public example.avro.User.Builder clearFavoriteNumber() {
      favorite_number = null;
      fieldSetFlags()[1] = false;
      return this;
    }

    /**
      * Gets the value of the 'favorite_color' field.
      * @return The value.
      */
    public java.lang.CharSequence getFavoriteColor() {
      return favorite_color;
    }

    /**
      * Sets the value of the 'favorite_color' field.
      * @param value The value of 'favorite_color'.
      * @return This builder.
      */
    public example.avro.User.Builder setFavoriteColor(java.lang.CharSequence value) {
      validate(fields()[2], value);
      this.favorite_color = value;
      fieldSetFlags()[2] = true;
      return this;
    }

    /**
      * Checks whether the 'favorite_color' field has been set.
      * @return True if the 'favorite_color' field has been set, false otherwise.
      */
    public boolean hasFavoriteColor() {
      return fieldSetFlags()[2];
    }

    /**
      * Clears the value of the 'favorite_color' field.
      * @return This builder.
      */
    public example.avro.User.Builder clearFavoriteColor() {
      favorite_color = null;
      fieldSetFlags()[2] = false;
      return this;
    }

    /**
      * Gets the value of the 'age' field.
      * @return The value.
      */
    public java.lang.Integer getAge() {
      return age;
    }

    /**
      * Sets the value of the 'age' field.
      * @param value The value of 'age'.
      * @return This builder.
      */
    public example.avro.User.Builder setAge(int value) {
      validate(fields()[3], value);
      this.age = value;
      fieldSetFlags()[3] = true;
      return this;
    }

    /**
      * Checks whether the 'age' field has been set.
      * @return True if the 'age' field has been set, false otherwise.
      */
    public boolean hasAge() {
      return fieldSetFlags()[3];
    }

    /**
      * Clears the value of the 'age' field.
      * @return This builder.
      */
    public example.avro.User.Builder clearAge() {
      fieldSetFlags()[3] = false;
      return this;
    }

    @Override
    @SuppressWarnings("unchecked")
    public User build() {
      try {
        User record = new User();
        record.name = fieldSetFlags()[0] ? this.name : (java.lang.CharSequence) defaultValue(fields()[0]);
        record.favorite_number = fieldSetFlags()[1] ? this.favorite_number : (java.lang.Integer) defaultValue(fields()[1]);
        record.favorite_color = fieldSetFlags()[2] ? this.favorite_color : (java.lang.CharSequence) defaultValue(fields()[2]);
        record.age = fieldSetFlags()[3] ? this.age : (java.lang.Integer) defaultValue(fields()[3]);
        return record;
      } catch (java.lang.Exception e) {
        throw new org.apache.avro.AvroRuntimeException(e);
      }
    }
  }

  @SuppressWarnings("unchecked")
  private static final org.apache.avro.io.DatumWriter<User>
    WRITER$ = (org.apache.avro.io.DatumWriter<User>)MODEL$.createDatumWriter(SCHEMA$);

  @Override public void writeExternal(java.io.ObjectOutput out)
    throws java.io.IOException {
    WRITER$.write(this, SpecificData.getEncoder(out));
  }

  @SuppressWarnings("unchecked")
  private static final org.apache.avro.io.DatumReader<User>
    READER$ = (org.apache.avro.io.DatumReader<User>)MODEL$.createDatumReader(SCHEMA$);

  @Override public void readExternal(java.io.ObjectInput in)
    throws java.io.IOException {
    READER$.read(this, SpecificData.getDecoder(in));
  }

}

@sijie That is the workflow you were worried about right? i.e.
schema A => generated pojo => ReflectData.parse => schema B

and schema A should equal schema B

@jerrypeng I got a different result than yours
Pulsar AllowNull     :{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":["null","string"],"default":null},{"name":"favorite_number","type":["null","int"],"default":null},{"name":"favorite_color","type":["null","string"],"default":null},{"name":"age","type":"int"}]}
Avro Not AllowNull   :{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}
Avro AllowNull       :{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}

@sijie I think the reason why AvroSchema.of(User.class) is producing a different result is because we shade all the Avro dependencies and that is interfering it from detecting that its a generated class, but  Avro Not AllowNull is the same as Avro Not AllowNull which we are using the exactly the later in AvroSchema.  Maybe we should figure out a way to getting the shading to work with Avro?

Here is a Test project i have created:
https://github.com/jerrypeng/TestAvro
I verified the reason why the AvroSchema part of the shaded pulsar-client library outputs a different schema is because of shading

@jerrypeng  correct. I added one more to complete the picture.
        log.info("Pulsar AllowNull                  :{}", new String(AvroSchema.of(User.class).getSchemaInfo().getSchema()));
        log.info("Pulsar shaded AVRO NOT AllowNull  :{}", org.apache.pulsar.shade.org.apache.avro.reflect.ReflectData.get().getSchema(User.class));
        log.info("Avro AllowNull                    :{}", ReflectData.AllowNull.get().getSchema(User.class));
        log.info("Avro Not AllowNull                :{}", ReflectData.get().getSchema(User.class));

Pulsar AllowNull                  :{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":["null","string"],"default":null},{"name":"favorite_number","type":["null","int"],"default":null},{"name":"favorite_color","type":["null","string"],"default":null},{"name":"age","type":"int"}]}
Pulsar shaded AVRO NOT AllowNull  :{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":"int"},{"name":"favorite_color","type":"string"},{"name":"age","type":"int"}]}
Avro AllowNull                    :{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}
Avro Not AllowNull                :{"type":"record","name":"User","namespace":"example.avro","fields":[{"name":"name","type":"string"},{"name":"favorite_number","type":["int","null"]},{"name":"favorite_color","type":"string"},{"name":"age","type":"int","default":19}]}

As you described above, AllowNull doesn't matter if the POJO is generated by AVRO, because AVRO will try to use SCHEMA$ field to retrieve the schema. But If the POJO is not generated by AVRO, AVRO will attempt to parse it and AllowNull will generating two different incompatible schemas.
my tests avoid covered all the cases.

Avro AllowNull and Avro Not AllowNull are using unshaded avro library to parse avro generated POJO. in this way, they will always produce the same schema no matter AllowNull used or not.

Pulsar AllowNull and Pulsar shade avro Not Allow Null are using shaded avro library to parse avro generated POJO. In this way, they will produce a completely different schema. AllowNull matters here.

The 2) indicates two problems:
a) for normal POJO (not generated by AVRO), AllowNull is producing incompatible schemas. so we need #3752
b) for avro generated POJO, shaded AVRO library (the one used by Pulsar) doesn't recognize it is an AVRO generated POJO. We need a separate fix to make sure Pulsar shaded AVRO working with avro generated POJO.

@sijie yup though I am not sure setting AllowNull is producing an incompatible schema.  Incompatible to what?  I guess if a user wrote some code:
ReflectData.get().getSchema(MyClass.class)

and they didn't set allow null then it wouldn't be compatible with the current schema generated from AvroSchema but my implementation could just have easily set allow null.  Thus, this doesn't seem like big deal.  I think the bigger problem here is the one with shading and how generated java classes are handled since that is a API/standard that should be preserved

yup though I am not sure setting AllowNull is producing an incompatible schema. Incompatible to what?

@jerrypeng ReflectData.AllowNull and ReflectData are generating two different incompatible schemas. I have been explaining to you guys in previous comments: my broader concern is NOT about generated POJO, is about using a same POJO class across many different systems using AVRO as the standard serialization format where people are using ReflectData not ReflectData.AllowNull (the second case explained at https://github.com/apache/pulsar/issues/3741#issuecomment-469506795).

my implementation could just have easily set allow null. Thus, this doesn't seem like big deal and arbitrary.

well it is a big deal when exchanging data between systems using same POJO.

Should we not preserve that semantic?

If Pulsar uses AVRO on POJOs, I don't think it is the responsibility for Pulsar to preserve its own semantic. It should preserve what AVRO preserves for everyone because this is the rules of all other systems follow when using AVRO. That's the most important thing when AVRO is used as a standard data format across data systems. Currently Pulsar's AllowNull implementation violates it.

but I think the bigger problem here is the one with shading

The shading bug is new to me. I agreed that we need to fix it.

Avro Schema compatability https://docs.confluent.io/current/schema-registry/docs/avro.html#compatibility-types


Why rename the main function to Java_java_security_AccessController_getProtectionDomain

When I decompile the java.exe(jdk 13) with ghidra 9.0.4 and ghidra  9.1.1, it's showing very strangely different results.

ghidra  9.1.1 showes WARNING：PDB signature does not match, but ghidra
9.0.4 is fine. I am pretty sure that the PDB signature match.
I do not find the main function in ghidra 9.1.1, and comparing the same position with main function
in ghidra 9.0.4 it is the Java_java_security_AccessController_getProtectionDomain function.
The Symbol Tree > Functions show very difference.

What java distribution is the java.exe from? I'd like to reproduce this.

Thanks. https://github.com/bahuwang/debugreport . The two screenshots show some difference.

Your java.exe indicates that the PDB should have GUID=c19971c4-3cd4-4b80-a78a-784723db1984 and AGE=1
Your java.pdb has GUID=FB36EAAC-86E8-4ABF-90B2-E84C697C6A3E and AGE=1.
In order for symbols to be laid down on your program correctly, these need to match.
Since the PDB does not match, it is almost a certainty that symbols will not line up with the addresses in the non-matched java.exe.
It is possible that you have the correct PDB somewhere else on your system, but it will likely try to use the one you have in the same folder as the java.exe file.  You can specify a symbol folder to search in the PDB Analyzer options.  Maybe that folder is specified in your Ghidra 9.0.4, but you have a different folder specified in 9.1.1.

I went to openjdk download site: https://github.com/ojdkbuild/ojdkbuild/releases.
There are two <IP_ADDRESS>-1 releases there, but no <IP_ADDRESS> at this point.
java-13-openjdk-debug-<IP_ADDRESS>-1 has java.exe and java.pdb.
java-13-openjdk-<IP_ADDRESS>-1 has a debug package (with java.pdb) that is separate from the main download that has the java.exe.
For each of these packages, the GUID of java.pdb and that specified by java.exe do not match. I wanted to assure myself that we did not introduce a bug in Ghidra, so I checked the GUIDs as specified by Ghidra 9.0.4, and they agree with Ghidra 9.1.1, and indicate that the PDB and executable do not match.  I'd also hate to believe that openjdk was not providing the correct PDB with their build, so I also downloaded this package as well: java-11-openjdk-debug-<IP_ADDRESS>-1.  It also shows has a java.pdb whose GUID does not match what is specified in java.exe.
I cannot, at this time, explain what is going on.

Thank you. At the beginning, I do not think about the difference of GUID and suppose they are the same, beacuse I build them.
The reason java.pdb do not match java.exe is that there has a some filename java.dll. The java.dll compile before the java.exe, and the GUID of java.dll match the java.dll.
It is not a bug for Ghidra 9.1.1, but I think Ghidra 9.0.4 may be a better fault tolerance or cleverer. Even if the GUID is not match, Ghidra 9.0.4 can load the PDB that map the exe file.
Now I do not know which is better, tightened inspection of Ghidra 9.1.1 or easy use of Ghidra 9.0.4.

Thank you. At the beginning, I do not think about the difference of GUID and suppose they are the same, beacuse I build them.
The reason java.pdb do not match java.exe is that there has a some filename java.dll. The java.dll compile before the java.exe, and the GUID of java.dll match the java.dll.
It is not a bug for Ghidra 9.1.1, but I think Ghidra 9.0.4 may be a better fault tolerance or cleverer. Even if the GUID is not match, Ghidra 9.0.4 can load the PDB that map the exe file.
Now I do not know which is better, tightened inspection of Ghidra 9.1.1 or easy use of Ghidra 9.0.4.

You can manually load the pdb file. To do so, in the code browser go to File->Load PDB... it should load it even if the signature doesn't match.

Certainly, by inspection, I can see that your java.pdb matches the java.dll and not the java.exe.
GUID=FB36EAAC-86E8-4ABF-90B2-E84C697C6A3E
AGE=1

@ghizard. Now I think Ghidra 9.1.1 has a wrong load path comparing Ghidra 9.0.4. For example when I analysis java.exe, its Program Information show the PDB File. But after my test, it first loads the same path of java.pdb with the java.exe, not the path of PDB File.
If there not has the file java.pdb at the same path, when analysising, it shows:
Unable to locate PDB file"e:\VSrepos\ojdkbuild\build\zip_debug\openjdk\support\native\java.base\java_objs\java.pdb" with matching GUID.
If there has the file java.pdb at the same path not matching the GUID, when analysising, it shows:
WARNING: PDB signature does not match.
It can test through https://github.com/bahuwang/debugreport.

One thing that changed in Ghidra 9.1 was the addition of the PDB analysis option to “not” search the PDB path that is specified in the PE header. See Issue #277.  The “default” of this option is different than how 9.0.4 worked, which used to allow Ghidra to reach out to a location that the user might not trust without the user having the ability to prevent it—it could be on an untrusted system. The option now prevents this, but allows the user to allow it anyways. This option was added as a quick solution providing safety, but could have been implemented in a more sophisticated fashion to only prevent UNC paths, but we would be presuming that all non-UNC paths are safe.
Since you built the target program and it’s PDB, then you would presumably trust them and the path locations in which they exist... thus you could change the option to allow the path to be searched.
To see the location that would be included in the search, you can, from the Project Manager, right click on java.exe and select About...  alternatively, you can do Help->About... in the Program Listing. With this view, you can locate where the PE header has indicated the PDB location.
There is one other thing I can think of that could be happening... the current mechanism for finding and verifying a PDB locates a putative match based on file names and paths... it only grabs the first possible one of these... and then it tries to validate it based on GUID and age. If those do not match, no other PDBs that could have been found in that combination of file names and paths are checked. (I have a newer mechanism that keeps and checks a list of these, instead of just one, but that code is not yet incorporated, and might get replaced with something even better before incorporation.). Note, too, that there is another PDB Analyzer option that could help Ghidra find the file.  It is the one that allows you to specify the symbols path. I don’t have it up in front of me at the moment to give you the exact name, but it is there.

I cannot recall if the path that is preferred first is the one that contains the executable. If that is the case, then another solution would be for you to temporarily move or rename the java.pdb file that is in the folder with the java.exe (but actually belongs to java.dll).


GTA V doesn't start with nvidia-drm.modeset=1

If I set nvidia-drm.modeset=0 then the game runs flawlessy, but at the same time I don't have vsync outside of the game. So I need to switch it on and off which is annoying.
If I set nvidia.drm.modeset=1 then I see "DirectX not found" error when I try to start GTA V.
Software information
GTA V (Steam), Ubuntu 18.10 (Gnome)
System information

GPU: GTX1070M
Driver: 415.27
Wine version: 4.0
DXVK version: 0.94 (Proton 3.16-6)

Make sure that Vulkan works on your end. This is not a DXVK bug, but enabling nvidia.drm.modeset=1 somehow broke your Vulkan setup.

@kakserpom I'm using 'nvidia.drm.modeset=1' and it works just fine. Please make sure that you are not running Wayland but X when using this switch ;)

@jp7677 how to make sure of that?

@kakserpom Please use something like https://unix.stackexchange.com/questions/202891/how-to-know-whether-wayland-or-x11-is-being-used to ensure what session you are running. If you are indeed using Wayland you can  just select "Gnome on Xorg" (or similar, don't remember exactly  what it is called) as session type when logging in.

Mmh, alright. False bet from my side then. The game though works with nvidia drm enabled, just did a test a minute ago. Something else is wrong on your system. Does the nvidia control panel works on your system with nvidia drm enabled?

@jp7677 Yes, it works just fine. I tried different driver version, I tried KDE. No luck. I could give you access to the laptop via TeamViewer :)

Do you have any other Vulkan application you could test? Both x64 and x86? 'vulkaninfo' might be a good start..

@jp7677 https://gist.github.com/kakserpom/adfe465fba42469f14e9ac5c52fe9ae0

https://bugs.launchpad.net/nvidia-drivers-ubuntu/+bug/1800465

I'm running Fedora and a desktop machine, I'm the wrong one to answer. Only solution I could think of is indeed disabling nvidia drm.

@Leopard1907 what driver are you on?

415.27


Add resolve dependency extension to LS

Purpose
Add resolve dependency extension to LS
Check List

[x] Read the Contributing Guide
[ ] Updated Change Log
[ ] Checked Tooling Support (#)
[ ] Added necessary tests

[ ] Unit Tests
[ ] Spec Conformance Tests
[ ] Integration Tests
[ ] Ballerina By Example Tests

[ ] Increased Test Coverage
[ ] Added necessary documentation

[ ] API documentation
[ ] Module documentation in Module.md files
[ ] Ballerina By Examples

@mohanvive / @IMS94 Do we need this PR? Looks like it got automatically closed without any attention. If you really don't need this PR anymore, please remove the stale label from the PR. Then it won't get added to the [Stale PR] Ballerina Stale PR Details mail in the future.

@anupama-pathirage we don't need this. We have introduced the online mode + the pull module code action to address these.


Steps are not in correct order in README

Description
In the 'How to Make a Pull Request?' section of the README file step 3 and step 4 are not in order.
Here we first need to go inside the CodingEasy folder and then we can set upstream.  Without going inside the folder of CodingEasy if we try to set upstream it gives an error 'fatal: not a git repository (or any of the parent directories): .git' .
Screenshots

Assigning you the same @Rushijaviya

@Rushijaviya  @KKhushhalR2405
I have noticed this issue but I made the right solution and continued to do my task.
And there is an error also which is exists in the command add. , the solution is should be space between the word 'add' and the '.' symbol

Actually, I have already finished the work. I was exploring a project that's why I haven't done PR.


Certificate pinning support

A common way to add ssl pinning support in iOS apps is to use the NSURLConnectionDelegate to define a custom didReceiveAuthenticationChallenge. How to add this security feature to Siesta framework?
For a complete example see the OWASP website.

By default, Siesta uses NSURLSession and not NSURLConnection, so look up instructions on how to do certificate pinning with an NSURLSessionDelegate instead of an NSURLConnectionDelegate.
Configure an NSURLSession with your custom NSURLSessionDelegate just as you would without Siesta. Then pass the NSURLSession as your networking provider when you create the service.
let certificatePinningSession = NSURLSession(
    configuration: NSURLSessionConfiguration.ephemeralSessionConfiguration(),
    delegate: MyCustomSessionPinningDelegate(),
    delegateQueue: nil)
let myService = Service(baseURL: "http://what.ever", networking: certificatePinningSession)

You might also take a look at the SSL conveniences provided by Alamofire (which you can also use with Siesta).
Finally — or I should say first! — consider using TrustKit instead of configuring it manually.


es7 decorators parse fail

https://github.com/wycats/javascript-decorators/blob/master/README.md
@abc({})
class ABC{  }

Unhandled rejection SyntaxError: Unexpected character '@' (1:0)

Our official policy on ES7 features is that we try to support all stage 4 proposals. Decorators is still at stage 1, and has the potential of being rejected. Therefore it is not considered at this moment.

I've added a note to clarify our ES7 status to the readme.


Socket Exception and error CS0246

I get three errors when I try to finish "Chapter 1: setting up the scene" by pressing play the first time.
(1): SocketException: The access to a socket was not allowed due to the access rights of the socket (translated from german):
System.Net.Sockets.Socket.Bind (System.Net.EndPoint localEP) (at <3845a180c26b4889bc2d47593a665814>:0)
SyntaxTree.VisualStudio.Unity.Messaging.UdpSocket.Bind (System.Net.IPAddress address, System.Int32 port) (at <7bfe50ea87ae444bb97944401234ceb7>:0)
SyntaxTree.VisualStudio.Unity.Messaging.Messager..ctor (System.Int32 port) (at <7bfe50ea87ae444bb97944401234ceb7>:0)
SyntaxTree.VisualStudio.Unity.Messaging.Messager.BindTo (System.Int32 port) (at <7bfe50ea87ae444bb97944401234ceb7>:0)
SyntaxTree.VisualStudio.Unity.Bridge.VisualStudioIntegration+<>c__DisplayClass9_0.<.cctor>b__0 () (at <843b7c5ab4d34b2491669c24f4d94522>:0)
SyntaxTree.VisualStudio.Unity.Bridge.Project+<>c__DisplayClass40_0.<RunOnceOnUpdate>b__0 () (at <843b7c5ab4d34b2491669c24f4d94522>:0)
UnityEditor.EditorApplication.Internal_CallUpdateFunctions () (at C:/buildslave/unity/build/Editor/Mono/EditorApplication.cs:200)
(2): Assets\Holograms\Support\Spatial Mapping\SpatialMapping.cs(24,13): error CS0246: The type or namespace name 'SpatialMappingRenderer' could not be found (are you missing a using directive or an assembly reference?)
(3): Assets\Holograms\Support\Spatial Mapping\SpatialMapping.cs(27,13): error CS0246: The type or namespace name 'SpatialMappingCollider' could not be found (are you missing a using directive or an assembly reference?)

Not sure how to fix any of them, as I'm beginner.

Dokumentdetails
⚠ Bearbeiten Sie diesen Abschnitt nicht. Er ist für die Verknüpfung von docs.microsoft.com zum GitHub-Artikel erforderlich.

ID: 9d680e05-a73a-321c-e7c8-8daa0a2c5249
Version Independent ID: 8ef33125-aa37-cc1a-0264-bba3a12daf5d
Content: MR Basics 101E - Complete project with emulator - Mixed Reality
Content Source: mixed-reality-docs/holograms-101e.md
Product: mixed-reality
GitHub Login: @keveleigh
Microsoft Alias: kurtie

For 2 and 3, likely the Windows Mixed Reality package didn't properly install itself. With some versions of Unity, it seems there's a dependency to either:

Toggle the XR Supported checkbox under UWP Player Settings
Go to Window -> Package Manager and ensure Windows Mixed Reality is downloaded and updated


Add meduza extension

Description
Another one news extension. Based on rss

Screencast
https://github.com/user-attachments/assets/7db64fc6-47dd-4a18-933b-97be1f560ef0

Checklist

[x] I read the extension guidelines
[x] I read the documentation about publishing
[x] I ran npm run build and tested this distribution build in Raycast
[x] I checked that files in the assets folder are used by the extension itself
[x] I checked that assets used by the README are placed outside of the metadata folder

I'm not sure why this one slipped my mind, very sorry about that 🫠

I think I would use useFetch from @raycast/utils since it has built-in caching and use parseResponse inside that to format the data - do you want an example of that?

Hello!
I fixed the "usefetch" issue and added a news summary feature


[ST] Delays on alUSD transfers from Ethereum to Optimism

Describe the bug
The transaction is going on for 4.5+ hours now. Is this a liquidity problem with Alchemix? Could be a similar situation to https://github.com/connext/monorepo/issues/5167 https://connextscan.io/tx/0xd923a3619b19db4a769977d9ea20bf0f4aea817cd9a85f1a9765970c9db57bb9
Link to ticket in discord
https://discord.com/channels/454734546869551114/1176320039218073662/1176320321696047164

Another one going on for 5.5+ hours https://connextscan.io/tx/0x7074efb2ce857a01e4c6be67b4e82f347edae6bf65ced61c574ea87435760773
https://discord.com/channels/454734546869551114/1176321729711321218/1176322251843457034

And another one for 3h https://connextscan.io/tx/0x0a5563cbb864c0d7522c3a5236e4d83d1a72a58f344c3be402ae7cb1f214e70a

All of the above have gone through. The cause was propagate failing because of gelato


pkg/assets/create: Replace Verbose and StdErr with Logger

To allow consumers to plug in their preferred logger instead of requring an output stream.  I'm using go-log/log as an unopinionated interface, which callers can feed with the stdlib's log package, logrus, etc. depending on their logging preferences.
CC @mfojtik, @sttts

We should not import new package here, instead we can create the interface here and plug whatever we need there (the interfaces are exchangeable). I would just copy https://github.com/go-log/log/blob/master/log.go#L5 at the top of the create.go

/hold
The whole org uses glog for very clear reason and because the whole k8s apimachinery and client-go is tightly coupled to it. Using more then 1 logger is a lost of time and will not work given the hardcoded glog in k8s. You just don't use 2 loggers in 1 project.

/cc @deads2k

Decoupling the code from glog is fine, but not by pulling in another package. Let's just abstract with a local interface and every consumer can plug a wrapper in for his/her favourite logger.

Making more loggers configurable is work that could have been better spent on important things, same goes for reviews.
In the end the same statement applies for logging in the end component, if it doesn't use glog.
(note: I personally don't like glog at all, but I understand the necessity of using it implied by being hardcoded in k8s.)

The whole org uses glog for very clear reason...

The installer inherited logrus.  I don't see it consuming this code directly, but I also don't see a need for opinionated logging in libraries.  However, I can reroll to take glog (or klog?) types if folks don't like the interface approach.

We should not import new package here, instead we can create the interface here and plug whatever we need there (the interfaces are exchangeable).

Why copy/paste this (and the no-op implementation), when they're already written and can be vendored?  Seems easier to maintain that way, and if your vendor tooling pruned unused packages it would be about the same amount of code either way.

Also, lots of Kubernetes library log handling discussion in kubernetes/kubernetes#61006, where interface-based approaches were raised but not accepted (although I'm still not clear on why not).

The installer inherited logrus. I don't see it consuming this code directly, but I also don't see a need for opinionated logging in libraries. However, I can reroll to take glog (or klog?) types if folks don't like the interface approach.

Klog will be brought up with 1.13 kubernetes, so I wouldn't bother with that, yet.

Also, lots of Kubernetes library log handling discussion in kubernetes/kubernetes#61006, where interface-based approaches were raised but not accepted (although I'm still not clear on why not).

That's one of the reasons klog happened, it might be a road towards just that, we'll see.

/hold

Hrm.  I went to reroll this to use glog as requested by @tnozicka here @deads2k here.  But I'm stuck with collecting the logs for unit tests.  Should I drop those tests?  Should I have init code set -log_dir and collect the test logs there?  Should I add an interface after all so the test suite can plug in a capture implementation (but with copies to local code, and not the vendored capture implementation I'm currently using as of 673e3d38ede?


Internal and External References - Remove "downloadable" filter

We can merge all internal (Assets) and external references (DocumentReferences) now.

Complete


tests: Clean cmath and climits usage

remove usage of limits.h that should not have existed
merge the two separate handling of nextafter direction
remove workaround for cmath impls with missing std::

deprecates PR #2443

My vague memory was that I had used the limits.h versions based on std:: portability issues with Android.  Building on our internal CI.  @cnorthrop  -- do you have bandwidth to do Android sanity testing on this?

@jzulauf-lunarg limits.h is completely unnecessery as we do not use plain int and unsigned.
But yeah, standard library standardness was a known problem on Android, and it affects <math.h> vs <cmath>
Thought it was time to start removing workarounds (as public CI seems fine with this now)...

Clean on our CI as well, I'd like to wait on Cody @cnorthrop  for the approval on this, just in case there are compile and target combinations CI isn't covering.
BTW -- change is a nice simplification.

Looks good from our end, although we're still at partial test coverage due to our office move.  We can revisit if it becomes a problem when we get remaining systems back online.


Number of qeuries for panoptic segmentation

According to the paper, there should be 400 (300 things and 100 stuff) queries for panoptic segmentation. However, I do not find this setting in the demo or evaluation codes. It seems that only 300 queries are used.

Yes, we have not released the code for using 400 queries. We will release it later.

We have released the decouple training code.


bug: Fix incorrect WebApi name.

Codecov Report

Merging #70 (1d37364) into main (40d631b) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #70   +/-   ##
=======================================
  Coverage   66.59%   66.59%
=======================================
  Files          31       31
  Lines         440      440
  Branches       17       17
=======================================
  Hits          293      293
  Misses        142      142
  Partials        5        5

:mega: Codecov can now indicate which changes are the most critical in Pull Requests. Learn more


Update GitHub Deployment Protection Rules quickstart

Summary
Fix bullet formatting in description and add documentation link

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.