Retire ACE1

[x] remove all dependencies on ACE1, ACE1x
[x] rewrite all tests to not require ACE1, ACE1x
[x] rewrite the documentation, possibly slimmed down initially.

Further Notes

I'm removing the fast experimental kernels (UltraFastACE) for now
Writing and reading of models will go into a separate PR
Export to LAMMPS will not be immediately updated.

close to ready to merging. But needs PRs in ExtXYZ and AtomsBase merged first.

@wcwitt  -- the tests are failing here, but in fact that's only because they rely on two PRs that haven't been merged yet, one in AtomsBase and one in ExtXYZ.
Tomorrow I want to add merge the JSON interface, convert the tutorials to using that interface, then add a test suite that confirms that the error tables from the ACEpotentials paper are reproduced. If this can all be done, then I think we may be able to do the tutorial on 0.8.


Link Instrument error

I received an error message when linking the instrument in a new station Belvedere Farm.  The instrument is now linked, but a link has also been made other places"
Site - a link has been made under organisations
Stations - a link has been made under organisations
Instruments - a link has been made under organisations

Link saving now checks for duplicates and prompts with a more useful error message.


[MSTest][Discovery] MSTestPlayground.exe Failed to discover tests from assembly (After upgrade from 2.1.2 to 2.2.8)

Description
Getting the following error while trying to run MSTests that are being hosted in a console application assembly
Microsoft (R) Test Execution Command Line Tool Version 16.11.0 Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait... A total of 1 test files matched the specified pattern. [MSTest][Discovery][MSTestPlayground.exe] Failed to discover tests from assembly MSTestPlayground.exe. Reason:It is illegal to reflect on the custom attributes of a Type loaded via ReflectionOnlyGetType (see Assembly.ReflectionOnly) -- use CustomAttributeData instead. No test is available in C:\Users\niboger\source\repos\MSTestPlayground\bin\Debug\MSTestPlayground.exe. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Additionally, path to test adapters can be specified using /TestAdapterPath command. Example /TestAdapterPath:.
Steps to reproduce
I've created this repo for reproduction:  https://github.com/nir-boger/ms-test-v2.2.8-bug
Expected behavior

1 Test being executed and passed

Actual behavior

No tests were executed

Environment

Win11
16.11.0
2.2.8

Same here, I'm trying to run unit tests within a project having exe output type. Using 2.1.2 everything works just fine, but with 2.2.7 and 2.2.8 I get the same error.

Same for me, 2.2.5 still works but from 2.2.6 on it's not working anymore.


Serve static html even have items in cart

Whenever customer has item in cart, nginx stops server html file. The idea is still serve it while exclude the cart. Do you think it's possible?

Any update on this, @maximejobin?

Your question is related to your theme and/or WP-Rocket. It has nothing to do with Rocket-Nginx. WP-Rocket already manages the fact that the cart should not be put in cache. If it's not in cache, Rocket-Nginx won't serve the static file as it won't exist.

Thank you for clarifying it. I'll ask them about this.


[ Feature Request ] Make the comment_marker customizable

The DEFAULT_COMMENT_MARKER serves as the unique identifier for PR comments, essential for determining which PR comment to update in replace mode. However, this approach becomes problematic when reporting multiple NYC code coverage results. Currently, each subsequent report overwrites the previous one, compelling users to opt for the new mode. This mode, while effective, can lead to an overload of comments in pull requests, a situation particularly cumbersome in monolithic architectures where, for example, ten services might generate ten new comments with each pipeline run. To mitigate this issue, making the comment marker customizable would be beneficial. By allowing each instantiation to define its own marker, we can utilize the replace mode more effectively, eliminating the risk of overwriting incorrect comments.

Would be super useful! I might try implementing myself (but don't wait up for me 😅)


added new cssWithContext utility

Ahoy hoy!
First time contributing long time utilizing. This is more of an idea with a basic implementation of the possible interface we could expose. Couldn't get the project running locally so I doubt it actually works
I am in the process if trying to make the following API possible:
const background = cssMixin`
  background-color: ${({isDark}) => isDark ? "black" : "white" };
`

const border = cssMixin`
  border: 1px solid black;
  border-radius: ${({curved}) => curved ? 5 : 1};
`

const MyComposedComponent = styled.div`
  display: block;
  ${background({isDark: false})}
  ${border({curved: true})}
`

const CoolContainer = () => (
  <MyComposedComponent isDark /> {/* isDark here will overwrite isDark as defined in the mixin utilization of the component */}
);

The code for cssMixin is as follows (its hard to read and probably horridly broken, hens why I'm making this PR)
import { isArray, isFunction } from "lodash";

const enhanceFunction = (interpolation, defaultValues) => {
  const functionWithEnhancedResult = (props) => {
    const functionResult = interpolation({ ...defaultValues, ...props });
    return enhance(functionResult, defaultValues);
  };

  return functionWithEnhancedResult;
};

export const enhance = (interpolation, defaultValues) => {
  if (isArray(interpolation)) {
    return interpolation.map(item => enhance(item, defaultValues));
  }

  if (isFunction(interpolation)) {
    return enhanceFunction(interpolation, defaultValues);
  }

  return interpolation;
};

export const cssMixin = (styles, ...interpolations) => (defaultValues = {}) => {
  return css.apply(null, [styles, ...enhance(interpolations, defaultValues)]);
};

Now this mostly works.. However I've been having an issue when I also use this with my cssIf helper:
import { css } from "styled-components";
import { isFunction } from "lodash";

const runConditionCheck = (condition, props) => {
  const isFunctionAndPasses = isFunction(condition) && condition(props);
  const isNotFunctionAndIsTruthy = !isFunction(condition) && !!condition;
  return isFunctionAndPasses || isNotFunctionAndIsTruthy;
};

/*
yes I've tried to use my cssMixin instead of css.apply to fix the ability to nest, didn't work
when I try to fix the nesting issue I will likely just need to fix somthing dumb here, but
I really dislike this code anyway so I would rather this was built into styled-components
*/
export const cssIf = (condition) => (styles, ...interpolations) => (props) => {
  if (runConditionCheck(condition, props)) {
    return css.apply(null, [styles, interpolations]);
  } else {
    return "";
  }
};

Now I can probably figure out through some fiddling why it's not a happy chappy and I probably will have to (if I use cssIf I cannot use a cssMixin inside of it) but this got me into the styled-components codebase looking for how you folk do the css function. Of which when I look at it it feels like the cssWithContext function I've build could almost remove any and all need for my horrendous  recursive enhance function and I could instead use:
export const cssMixin = (styles, ...interpolations) => (defaultValues = {}) => cssWithContext(defaultValues)(styles, ...interpolations);

which would be much nicer. However I am looking at the code more and more and I don't think my PR would achieve what I want given I am causing the code to go down a different path that normal as it appears that css the helper doesn't inject any context into the functions inside of my template literals (see line 88 of flatten.ts, no executionContext is passed in when css used, thus it would just return the chunk without adding any of my custom state + there is no code to merge the correct executionContext with my custom context so even if this did work it would kill the context from other sources, as such my isDark attribute wouldn't come through if it even rendered at all)  (Sorry I tried to test but the sandbox project isn't working on my machine (Windows 10))
so I was wondering..
With my code example above of how I currently am achieving this to somewhat some success AND with my mock up of how I would love to expose this functionality nativly could anyone perhapse shed any light on how I could successfully.
OR
Is there some secret hidden mojo that makes this possible currently as is without a PR? All of the searching I've done has led me to the concision that my mixins either need to have no configurability (full access to the theme/attributes via my template functions like normal, but I cannot allow the implementing styled-compoenent of the mixin to define configuration ON the call to the mixin) OR I have to use a mixin that is a template litteral in itself, but has no access to the styled-compoenents context so I cannot access attribute data for the styled-component the mixin is being used inside of (isDark would stop working).
Cheers 👍 love the project, its a long post so no offense if this is a bit much for yall to handle.

further apologies if this was the incorrect way to bring this discussion up and if I've broken the build for this example, I couldnt get tests to pass on my machine, however allot of little things don't work as I guess all the developers are on MacOs/Linux

Hiya :wave: Sorry for the late reply. typically we don't accept proposals in PR form, if it a discussion didn't precede it. So sorry for not getting to this earlier. Basically, I don't think we can accept this. We get a lot of proposals for lots of kinds of API additions for either edge case uses, cases where workarounds exist already, or APIs that simply introduce "more" ways of doing one thing, which increases the load on beginners since there seizes to be a single way of doing things that's coherent.
Basically, in this case it's the last of these points. I've seen a lot of repos that have some form of () => css() usage, and while that's not quite the same, I'm very hestitant to introduce an API just to enable a syntax change. I'll go ahead and close this for now, but happy to discuss this, especially with a larger RFC and idea behind it.


KMM New Memory Model

@ychescale9 I see that you enabled the new memory model in a previous commit. Do you plan to only support the new memory model in the future or do you wish to support both?
In the new memory model there would no longer be a need for IsoMap. Just safe concurrent access.
I'd be happy to work on an implementation if you let me know which route you were intending to take.

I originally planned to only support new mm but if it's not too much effort to support both until new mm is stable that'd be better😀
I assume we just need to check the isNewMM flag in stdlib?
Super happy to accept PRs!

The most basic support would be enough to just check the isNewMM and then use a regular map or an isoMap depending on the case.. but there's probably other optimizations that could be made.
I'll have a look and see if we can do easy support for both without compromising the benefits of of the new MM.

Closing this as PR addressing this has been merged


Alignment Issue

Hi, this is just a minor issue but I guess you may want to acknowledge about it.
The vertical alignment of titleForEmptyDataSet, descriptionForEmptyDataSet and imageForEmptyDataSet appears differently in these scenarios:

The app is first launched
How it looks after 1 row was added and deleted

3 & 4. When moves to another screen (Show segue) and goes back.
Here is the overview of them:

I hope you already have the solution for this.

Thank you, I'm using your suggestion for now. Hope you will find a way to fix it soon :+1:

I found that this might have something to do with the topLayoutGuide aka the navigation bar and contentInset. It seems for me when the app is first loaded its moved down exactly the amount of topLayoutGuide.length.
I've been using some combination of the following but it's NOT consistent, most times it works, some times it still doesn't
    func emptyDataSetWillAppear(scrollView: UIScrollView!) {
        automaticallyAdjustsScrollViewInsets = false
        scrollView.contentInset = UIEdgeInsetsMake(0, 0, 0, 0)
        scrollView.contentOffset = CGPointMake(0, 0)
    }

    func emptyDataSetWillDisappear(scrollView: UIScrollView!) {
        automaticallyAdjustsScrollViewInsets = true
        collectionView?.contentInset = UIEdgeInsetsMake(topLayoutGuide.length, 0, 0, 0)
        collectionView?.contentOffset = CGPointMake(0, topLayoutGuide.length)
    }

Hope the insight helps, could really use a fix for this too

Sweet thanks! Can't believe this was discussed a whole 9 months ago.
The second comment you linked fixed it for me and has provided the most consistent experience. Code below
- (void)viewDidLayoutSubviews {
    [super viewDidLayoutSubviews];

    if (!self.tableView.emptyDataSetSource) {
        self.tableView.emptyDataSetSource = self;
        self.tableView.emptyDataSetDelegate = self;
        [self.tableView reloadEmptyDataSet];
    }
}

I've noticed that I'm only getting this behavior on the iPad. For now, I'm doing the following to get around this:
func verticalOffsetForEmptyDataSet(scrollView: UIScrollView!) -> CGFloat {
    if UIDevice.currentDevice().userInterfaceIdiom == .Pad {
        return -65
    }

    return 0
}

Yes, my viewDidLayoutSubviews is almost identical.

Hi,
None of the solutions suggested at working for me. I am using latest version of the framework.
What could be wrong?

i updated my pods but version is same as before ie 1.8.1 so didnt find it fixed.


NOTES: Create form to add a new note

Let's create a form that hits notes#create and creates a new note on a pregnancy. Nothing too fancy, just a blank text area.
Should be inserted into the partial mentioned in #187 .

@ajohnson052 grinding on this

Fixed via @ajohnson052 's work in #236


[1.2.25] Cannot exit command execution using ctrl+c

Trying to exit commands using ctrl+c does not work.
For example:
❯ rosa logs uninstall -c ci-rosa --watch
W: Logs for cluster 'ci-rosa' are not available
-
|
-
\ time="2023-08-23T05:36:11Z" level=debug msg="no additional log fields found"
...
...
time="2023-08-23T05:36:13Z" level=debug msg="search for IAM roles"
time="2023-08-23T05:36:13Z" level=debug msg="iterating over a page of 100 IAM roles"
^C
^C^C^C^C^C^C^C

^C^C^C^C^Ctime="2023-08-23T05:36:17Z" level=debug msg="iterating over a page of 100 IAM roles"
time="2023-08-23T05:36:20Z" level=debug msg="iterating over a page of 38 IAM roles"
time="2023-08-23T05:36:21Z" level=debug msg="search for IAM users"

Closing; cannot reporoduce


Unexpected requests to IAM in AWS region us-east-1

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug
Hi, we are operate an NodeJS Service in a VPC with restricted internet access. We configured the region eu-central-1 to which we have access, but sometime we can see the following errors in our logs:

It seems, that this is a request to a AWS Service (perhaps IAM) hosted in us-east-1.
Did we miss any required configuration?
We configured following environment variables:
AWS_REGION=eu-central-1
AWS_STS_REGIONAL_ENDPOINTS=regional
AWS_DEFAULT_REGION=eu-central-1
And we configured the region on the call of generateAuthToken.
aws-msk-iam-sasl-signer-js library version used
1.0.0
Which Node.js version is this issue in?
v18.18.2
Operating System and version
Alpine 3.18.4
Reproduction Steps
We send messages to Kafka in intervals of 30 seconds
Observed Behavior
Sometimes we see timeout errors in the log when the service tries to reach a service in us-east-1.
Expected Behavior
No timeout errors should be in the log.
Possible Solution
No response
Additional Information/Context
No response

Thank you for raising this. Is this blocking you from producing/consuming to MSK clusters?
Can you share the code snippet used in your client app to call the generateAuthToken method?

Hi! We ran into the same problem, and after setting up a minimal example with EKS, I’ve confirmed the problem.
generateAuthToken uses fromNodeProviderChain without passing in a client config for STS, and the AWS SDK v3 doesn’t support using the AWS_STS_REGIONAL_ENDPOINTS=regional environment variable.
I fixed this by calling generateAuthTokenFromCredentialsProvider to give a bit more control around the calls.
const AWS_REGION = 'us-east-2'

const log = (...content) => console.log(content[0])

async function oauthBearerTokenProvider(region) {
   // Uses AWS Default Credentials Provider Chain to fetch credentials
   const authTokenResponse = await generateAuthTokenFromCredentialsProvider({
           region,
           awsCredentialsProvider: fromNodeProviderChain({
             maxRetries: 3,
             clientConfig: {
               region,
               logger: {
                 trace: log, warn: log, info: log, error: log, debug: log
               }
             }
           })
   })

   return {
       value: authTokenResponse.token
   }
}

This also has the added benefit of allowing you to add a logger to the STS client used.
Hopefully this helps!


Fix _Set regression in functions handling

Closes #1987

IMHO, this deserves a prompt hot fix, either as 0.23 if there is enough on master, or as 0.22.2.

There is quite a bit on master, :+1:  for v0.23.0

I unfortunately will not have much time this weekend.  But if I manage to find the time and no one has beat me to it, I will try to get 0.23.0 out.

I will also look over this fix more closely.  It looks simple enough, but I don't know the _Set code very well.
@scott-christopher: if you have a few minutes, would you be willing to check into this?

Yep, apologies for the bug. This LGTM.

Thanks, Scott.  As expected, I did not find the time for a release over the weekend.  But I will try to do so soon.


Generate specs from TGI openapi.json

From an idea mentioned by @OlivierDehaene and @drbh in https://github.com/huggingface/huggingface.js/pull/579 and slack thread (internal).
This PR adds a script  inference-tgi-import.ts to generate the text-generation and chat_completion specs from the auto-generated TGI specifications. The goal is to keep in sync TGI improvements with @huggingface/tasks and therefore have a consistent "single source of truth". The converted specs are then compatible with our tooling to generate JS/Python code.
This PR changes quite a lot of naming in the generated JS/Python types. Luckily I don't think they were yet used in the JS ecosystem so better to do that now than later.
I also opened https://github.com/huggingface/huggingface_hub/pull/2237 to include these changes in huggingface_hub.
TODO:

[ ] fix lint errors. How to deal with a parsed json to avoid using any?
[ ] CI workflow to open a PR each time TGI is updated? => can be done in a future PR

Does this PR supersede #579?

We are losing the stream parameter in the input types - not sure if it's a big deal though?

@SBrandeis  Fixed in https://github.com/huggingface/huggingface.js/pull/629/commits/97934afe33cc1739d3af57e524ff32d8ba3b4f96 by taking definition from CompatGenerateRequest instead of GenerateRequest (see TGI code).

@SBrandeis I addressed (almost) all comments. For the not-enough explicit Type type, I'm a bit lazy to go for a solution in guicktype directly. Maybe you'll have another idea to fix it more easily? We could have a ad-hoc rule but I'm not a big fan of it :grimacing:
About docs for text_generation inputs, https://github.com/huggingface/text-generation-inference/pull/1798 has been merged but we'll have to wait for a release of TGI to get the updated openapi.json. In the meantime, I think it's fine not to have them documented and we'll fix that in a later PR (could even be automatized).
Otherwise, PR is almost good to review/merge I think. I still have the lint checks yelling at me but I don't have a good solution to deal with Any/unknown data without specify all attributes. Any suggestions? (I'd be fine disabling the rule for the entire file if that's possible TBH).

@SBrandeis can @Wauplin go ahead and merge the last changes? this PR will be very helpful for a chat completion method we're working on thanks

Technically there is still the lint issue. Any advice? :pray:

PR is almost good to review/merge I think. I still have the lint checks yelling at me but I don't have a good solution to deal with Any/unknown data without specify all attributes. Any suggestions? (I'd be fine disabling the rule for the entire file if that's possible TBH).

@Wauplin pushed a new version of the script that should pass the linter test - can you please double-check the logic is unchanged?
ping @coyotte508 for review as well - please let us know if you see anything that should not hit the codebase

Thanks for the changes @SBrandeis! Had a check to them and logic seems exactly the same yes :+1:
I pushed https://github.com/huggingface/huggingface.js/pull/629/commits/52ec1e8ece0ca24aa2a2ecfeff3ee36093c1cee3 for prettier changes but that's all. Let's wait for a final review and then we should be able to merge :)

CI is green so time to merge! :smile: Thanks @SBrandeis @coyotte508 for the reviews.
@radames feel free to use those types in inference.js now


Invalid SQL generated for multiple joins

Here is the simplified version of what I am executing - i removed other tables and left only Account
from a1 in c.Accounts
from a2 in c.Accounts.Where(a2=>a2.Id == a1.Id)
from a3 in c.Accounts.Where(a3 => a3.Id == a1.Id)
where a1.Id == id
select new { a1, a2, a3 }

The SQL generated is this
SELECT TOP(1) [a1].[Id], [a2].[Id]
FROM [account] AS [a1]
CROSS JOIN [account] AS [a2]
WHERE ([a2].[Pk] = [a1].[Pk]) AND ([a3].[Pk] = [a1].[Pk]) AND ([a1].[Pk] = @__8__locals1_pkAccount_0)

It is obviously invalid - a3 is included in WHERE clause but nowhere else.
I could have moved WHERE clauses to the linq where section but in the real life I would like to do LEFT JOINs like this (which worked fine in EF6)
from a1 in c.Accounts
from a2 in c.Accounts.Where(a2=>a2.Id == a1.Id).DefaultIfEmpty()
from a3 in c.Accounts.Where(a3 => a3.Id == a1.Id).DefaultIfEmpty()
where a1.Id == id
select new { a1, a2, a3 }

This fails with an exception
System.InvalidOperationException: There is already an open DataReader associated with this Command which must be closed first.

The first example works in current bits, producing the following TSQL
SELECT [a1].[Id], [a2].[Id], [a3].[Id]
FROM [Account] AS [a1]
CROSS JOIN [Account] AS [a2]
CROSS JOIN [Account] AS [a3]
WHERE (([a2].[Id] = [a1].[Id]) AND ([a3].[Id] = [a1].[Id])) AND ([a1].[Id] = @__id_0)',N'@__id_0 int',@__id_0=1

The second example also works, but produces very inefficient queries, basically the joins happen on the client. If you want to produce LEFT JOINs you can follow this pattern (GroupJoin + SelectMany + DefaultIfEmpty) recognized by EF:
                var query = from a1 in ctx.Accounts
                            join a2 in ctx.Accounts on a1.Id equals a2.Id into g1
                            from a2 in g1.DefaultIfEmpty()
                            select new { a1, a2, };

or in method syntax:
                var query = ctx.Accounts
                    .GroupJoin(
                        ctx.Accounts,
                        a1 => a1.Id,
                        a2 => a2.Id,
                        (a1, g1) => new { a1, g1 })
                    .SelectMany(
                        a2 => a2.g1.DefaultIfEmpty(),
                        (g, a2) => new { g.a1, a2 });

what if i want to left join with an operator other than equals? is that still possible? and why such a regression? I would say lots of code will have troubles migrating to ef7 because of it.

@MaximBalaganskiy unfortunately EF7 only supports joins with equality, so the scenario you are describing is not possible at the moment. Upon initial release, EF7 is not intended to be a recommended upgrade for everyone who is using EF6 at the moment. Consider http://blogs.msdn.com/b/adonet/archive/2014/12/02/ef7-priorities-focus-and-initial-release.aspx for more context on that matter.

Filed an issue to introduce those join patterns to EF7:
https://github.com/aspnet/EntityFramework/issues/3263

Wonderful, thanks for that.


怎么加群

New Issue Checklist

[ ] I have read the Documentation
[ ] I have searched for a similar issue in the project and found none

Issue Info

Info
Value

Platform Name
e.g. iOS / macOS

Platform Version
e.g. 12.0 / 10.14.0

TABAnimated Version
e.g. 5.0.0 / 4.4.0

Xcode Version
e.g. Xcode 11 / Xcode 10

Repro rate
e.g. all the time (100%) / sometimes x% / only once

Repro with our demo prj
e.g. does it happen with our demo project?

Demo project link
e.g. link to a demo project that highlights the issue

Issue Description and Steps
Please fill in the detailed description of the issue (full output of any stack trace, compiler error, ...) and the steps to reproduce the issue.

过期了

manager.animations()?(0,1)?.remove()
不能这样写啊

自行更新写法


Use into a DDEV Project. Is it possible?

I find this module really useful to add testing capabilities and I want to use it for my project.
However I followed the instructions and I realized that it won't work in my project because I'm already using DDEV in the project I'm working on.
Also, the 4th step ddev configure....  shows the ddev error:
"it usually does not make sense to ddev configin a subdirectory of an existing project. Is it possible you wanted toddev config in parent directory ...?"
As far I understand this project is not intended to be used into a DDEV project or am I wrong and is there then any way to use this project into a DDEV project?
Thanks in advance
Regards

You have to start with a clone of the contrib project. That must be in the root of the project for this add-on to work.

Aha,
I understand it.
As I supposed I can't just add this tool to my current project but I have to start a new one without DDEV an then use this tool in that project.
Thanks for the explanation.


time is not correctly updated under wine

After golang had changed its clocks to monotonic, windows port started to use slightly unofficial memory mapping hack to get current time: https://codereview.appspot.com/108700045/patch/120001/130001
Unfortunately wine (as of jan 2016 neither hack was committed) does not update that page, this ends up with time.Sleep() taking forever on wine (checked with GOOS=windows and go 1.7.4)
It seems that hack to read mapped memory containing interrupt time works for all systems but wine, are there any plans to add a check into magic systime() call at https://github.com/golang/go/blob/master/src/runtime/os_windows.go#L594 to fallback to query performance counter syscall for example if mapped memory is not being updated?

@dvyukov Dmitry please comment on this issue

@alexbrainman
Is it possible to detect that we are running under wine?

@dvyukov
Yes, there is a way, Daz's Windows Loader can detect if it's launched under wine somehow

This should work
#include <windows.h>
#include <stdio.h>

int main(void)
{
	static const char * (CDECL *pwine_get_version)(void);
	HMODULE hntdll = GetModuleHandle("ntdll.dll");
	if(!hntdll) {
		printf("Not running on NT.\n");
		return -1;
	}
	pwine_get_version = (const char* (*)())(void *)GetProcAddress(hntdll, "wine_get_version");
	if(pwine_get_version) {
		printf("Running on Wine... %s\n",pwine_get_version());
	} else {
		printf("did not detect Wine.\n");
	}
	return 0;
}

$ x86_64-w64-mingw32-gcc /tmp/wine.cpp -o /tmp/wine
$ /tmp/wine
fixme:winediag:start_process Wine Staging 2.0-rc2 is a testing version containing experimental patches.
...
Running on Wine... 2.0-rc2

But I believe that detecting that interrupt time is not updated rather than application runs under wine is a better idea - one problem, one local fix. Who knows, maybe wine will eventually port this memory feature, although so far there are only workarounds which use dedicated thread which updates this page once in 15.6 ms to mimic only system time.

@bioothod
Sure, we can change runtime to allow for Wine.
Searching for wine_get_version functoin in ntdll.dll on startup sounds good to me. I am not clear about your alternative proposal - how are you going to detect that "interrupt time is not updated"?
Also, once we know that we are running on Wine, how should runtime.unixnano and runtime.nanotime be implemented? Alternatively fill free to send your own proposal (https://golang.org/doc/contribute.html).
Thank you.
Alex

run this interrupt time checking loop in osinit() and if time doesn't change, set a flag/callback which uses modkernel32.NewProc("Sleep")

I do not understand your proposal. Do you suggest we check if runtime._KSYSTEM_TIME at runtime._INTERRUPT_TIME address changes in a loop? How many time should we run the loop before giving up? This should be fast - we don't want every Go program pay for this too much.
Alex

Interrupt time precision is about 10 updates per microsecond, so calling something like code below at every start will not ever slow down any go program, getting that windows starting code already calls things like LoadLibrary()
It could be even better than what I wrote if using a systime()-like loop directly instead of calling systime() which synchronizes timer update with timer interrupt. There should be a new term for this kind of optimization though - a nanooptimization!
I would not mind calling this with the loop duration specified via some environment variable or anything else, which will be zero by default and only wine users would specify it. Although imo it looks a bit ugly like hell, it is up to the maintainers who believes what is best for the project.
// 1 usec at worst
func time_check() bool {
t1 := systime()
for i := 0; i<10; i++ {
    t2 := systime()
    if t2 != t1 {
        return true
    }
}
return false
}

osinit() {
...
if time_check() != true {
    set new time callback
}
}

Interrupt time precision is about 10 updates per microsecond, so calling something like code below at every start will not ever slow down any go program

Fair enough. We could, probably, do that. Or search for wine_get_version in ntdll.dll.
You didn't say how runtime.unixnano and runtime.nanotime should be implemented for Wine.
Alex

I'm still not convinced.
Say, if ReactOS requires special cases, should we add them? Why wine is
different?
Wine is supposed to emulate Windows faithfully. Just because we generally
don't add special case for qemu-user, I don't think we should add special
case for wine too.
It's definitely possible to implement the system page under wine.

Why wine is
different?

The way runtime.unixnano and runtime.nanotime are implemented is not documented by Microsoft. I think it is fare we implement them in a documented way.
Personally I am happy to support whatever OS users use.
Alex

To implement nanotime in a correct fashion I would recommend using QueryPerformanceCounter/QueryPerformanceFrequency syscalls, dividing one by another should produce incrementing counter with fixed frequency: https://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx
Wine is a bit different, because it is not really a windows, it tries its best to emulate it, but it quite frequently fails. It is up to the wine user to convince wine core team to fix it, and frankly saying I seriously doubt wine can ever fix this particular bug for at least 1ms resolution. This bug exists in wine bugzilla for at least 2 years, the only solution I saw uses dedicated thread to update the mapping, it emulates system time with 15.6 ms resulution, which is not enough for go (although its ok for me and most of the users I believe).
Quite contrary syscall behaviour is a standard, if QPC breaks there will be no hpet timers, and I'm pretty sure wine needs it. It has quite a serious performance overhead compared to reading 3 times from mapped memory, so it is not default.
As of ReactOS - it doesn't really matter. It looks really feasible for me that some reactos user will eventually come here and say, that in version X.Y.Z of branch N something had stopped working. If golang can fix it easily without overhead, what's the problem?  And fixing some issue for some users with a trivial check is a good way to go. Otherwise you can tell that time subsystem (and I question whether things like GC work) virtually doesn't work on one of the most popular crossover platform in the world.
And after all this memory reading feature IS NOT OFFICIALLY DOCUMENTED. It exists in winapi headers quite for a while and starting from w2k there are some helpers which work with it (like is-debugger-present and similar), but yet again, if you fight for the purification, you should argue to change code to official syscalls.

To implement nanotime in a correct fashion I would recommend using QueryPerformanceCounter/QueryPerformanceFrequency syscalls

Sounds like a plan. Would you like to send a change? Here https://golang.org/doc/contribute.html is how to contribute.
Maybe wait for a week to make sure no objections here. @minux might come around too.
Alex

The hard part of wine is implementing the undocumented side effects. If
wine just implements documented interfaces, I doubt it can run any major
Windows applications at all.

Changing our implementation to use documented interfaces (provided the
performance doesn't suffer too much) SGTM. But detecting wine and then act
differently is not ok from me.

I've cooked up a patch, which kind of works but raises an error when calling time.Sleep(): fatal error: runtime: stack split at bad time
It happens 100% when time.Sleep() is invoked and never when just time.Now() or friends (which are different now compared to the same startup value previously).
If there anything with signals which messes up between time.Sleep() and QueryPerformanceCounter()?
Here is a patch, its quite straightforward
wine.time.use.qpc.diff.txt
Here is a tracedump:
$ GOTRACEBACK=2 /tmp/sleep
fixme:process:SetProcessPriorityBoost (0xffffffff,1): stub
t1: 2017-01-14 09:32:54.4705399 +0300 AST [1484375574470539900]
t2: 2017-01-14 09:32:54.4705495 +0300 AST [1484375574470549500]
2017-01-14 09:32:54.4705519 +0300 AST [1484375574470551900]: going to sleep at start for 1s
runtime: newstack sp=0x5b411f1c stack=[0x5b411000, 0x5b412000]
	morebuf={pc:0x421aad sp:0x5b411f20 lr:0x0}
	sched={pc:0x43ffa1 sp:0x5b411f1c lr:0x0 ctxt:0x0}
runtime.systime_qpc_syscall(0x7ffe0008, 0x0, 0x1)
	/home/zbr/awork/gosrc/src/runtime/os_windows.go:657 +0x7d fp=0x5b411f4c sp=0x5b411f20
runtime.nanotime(0x4ee13c, 0x0)
	/home/zbr/awork/gosrc/src/runtime/os_windows.go:667 +0x14 fp=0x5b411f5c sp=0x5b411f4c
runtime.notetsleep_internal(0x4ee13c, 0x3b9abe48, 0x0, 0x5b42e1e0, 0x0, 0x0, 0x5b42e1e0)
	/home/zbr/awork/gosrc/src/runtime/lock_sema.go:194 +0x6e fp=0x5b411f74 sp=0x5b411f5c
runtime.notetsleepg(0x4ee13c, 0x3b9abe48, 0x0, 0x0)
	/home/zbr/awork/gosrc/src/runtime/lock_sema.go:258 +0x77 fp=0x5b411f98 sp=0x5b411f74
runtime.timerproc()
	/home/zbr/awork/gosrc/src/runtime/time.go:209 +0x39b fp=0x5b411ff0 sp=0x5b411f98
runtime.goexit()
	/home/zbr/awork/gosrc/src/runtime/asm_386.s:1629 +0x1 fp=0x5b411ff4 sp=0x5b411ff0
created by runtime.addtimerLocked
	/home/zbr/awork/gosrc/src/runtime/time.go:116 +0xe0
fatal error: runtime: stack split at bad time

runtime stack:
runtime.throw(0x4a465a, 0x20)
	/home/zbr/awork/gosrc/src/runtime/panic.go:596 +0x7c fp=0x33fcbc sp=0x33fcb0
runtime.newstack(0x0)
	/home/zbr/awork/gosrc/src/runtime/stack.go:989 +0x9cd fp=0x33fd74 sp=0x33fcbc
runtime.morestack()
	/home/zbr/awork/gosrc/src/runtime/asm_386.s:409 +0x77 fp=0x33fd7c sp=0x33fd74

goroutine 18 [syscall]:
runtime.systime_qpc_syscall(0x7ffe0008, 0x0, 0x1)
	/home/zbr/awork/gosrc/src/runtime/os_windows.go:657 +0x7d fp=0x5b411f4c sp=0x5b411f20
runtime.nanotime(0x4ee13c, 0x0)
	/home/zbr/awork/gosrc/src/runtime/os_windows.go:667 +0x14 fp=0x5b411f5c sp=0x5b411f4c
runtime.notetsleep_internal(0x4ee13c, 0x3b9abe48, 0x0, 0x5b42e1e0, 0x0, 0x0, 0x5b42e1e0)
	/home/zbr/awork/gosrc/src/runtime/lock_sema.go:194 +0x6e fp=0x5b411f74 sp=0x5b411f5c
runtime.notetsleepg(0x4ee13c, 0x3b9abe48, 0x0, 0x0)
	/home/zbr/awork/gosrc/src/runtime/lock_sema.go:258 +0x77 fp=0x5b411f98 sp=0x5b411f74
runtime.timerproc()
	/home/zbr/awork/gosrc/src/runtime/time.go:209 +0x39b fp=0x5b411ff0 sp=0x5b411f98
runtime.goexit()
	/home/zbr/awork/gosrc/src/runtime/asm_386.s:1629 +0x1 fp=0x5b411ff4 sp=0x5b411ff0
created by runtime.addtimerLocked
	/home/zbr/awork/gosrc/src/runtime/time.go:116 +0xe0

goroutine 1 [sleep]:
runtime.gopark(0x4a6d40, 0x4ee130, 0x49f53a, 0x5, 0x13, 0x2)
	/home/zbr/awork/gosrc/src/runtime/proc.go:261 +0x109 fp=0x5b429ea0 sp=0x5b429e8c
runtime.goparkunlock(0x4ee130, 0x49f53a, 0x5, 0x13, 0x2)
	/home/zbr/awork/gosrc/src/runtime/proc.go:267 +0x4c fp=0x5b429ebc sp=0x5b429ea0
time.Sleep(0x3b9aca00, 0x0)
	/home/zbr/awork/gosrc/src/runtime/time.go:59 +0xfb fp=0x5b429ed8 sp=0x5b429ebc
main.main()
	/tmp/sleep.go:18 +0x466 fp=0x5b429fc8 sp=0x5b429ed8
runtime.main()
	/home/zbr/awork/gosrc/src/runtime/proc.go:185 +0x1d6 fp=0x5b429ff0 sp=0x5b429fc8
runtime.goexit()
	/home/zbr/awork/gosrc/src/runtime/asm_386.s:1629 +0x1 fp=0x5b429ff4 sp=0x5b429ff0

goroutine 2 [force gc (idle)]:
runtime.gopark(0x4a6d40, 0x4edfe8, 0x4a08de, 0xf, 0x2914, 0x1)
	/home/zbr/awork/gosrc/src/runtime/proc.go:261 +0x109 fp=0x5b414fbc sp=0x5b414fa8
runtime.goparkunlock(0x4edfe8, 0x4a08de, 0xf, 0x5b40e014, 0x1)
	/home/zbr/awork/gosrc/src/runtime/proc.go:267 +0x4c fp=0x5b414fd8 sp=0x5b414fbc
runtime.forcegchelper()
	/home/zbr/awork/gosrc/src/runtime/proc.go:226 +0x99 fp=0x5b414ff0 sp=0x5b414fd8
runtime.goexit()
	/home/zbr/awork/gosrc/src/runtime/asm_386.s:1629 +0x1 fp=0x5b414ff4 sp=0x5b414ff0
created by runtime.init.3
	/home/zbr/awork/gosrc/src/runtime/proc.go:215 +0x2b

goroutine 3 [GC sweep wait]:
runtime.gopark(0x4a6d40, 0x4ee090, 0x4a030a, 0xd, 0x419014, 0x1)
	/home/zbr/awork/gosrc/src/runtime/proc.go:261 +0x109 fp=0x5b415fb0 sp=0x5b415f9c
runtime.goparkunlock(0x4ee090, 0x4a030a, 0xd, 0x14, 0x1)
	/home/zbr/awork/gosrc/src/runtime/proc.go:267 +0x4c fp=0x5b415fcc sp=0x5b415fb0
runtime.bgsweep(0x5b4041c0)
	/home/zbr/awork/gosrc/src/runtime/mgcsweep.go:56 +0xa2 fp=0x5b415fe8 sp=0x5b415fcc
runtime.goexit()
	/home/zbr/awork/gosrc/src/runtime/asm_386.s:1629 +0x1 fp=0x5b415fec sp=0x5b415fe8
created by runtime.gcenable
	/home/zbr/awork/gosrc/src/runtime/mgc.go:212 +0x55

goroutine 17 [finalizer wait]:
runtime.gopark(0x4a6d40, 0x4fde30, 0x4a0628, 0xe, 0x14, 0x1)
	/home/zbr/awork/gosrc/src/runtime/proc.go:261 +0x109 fp=0x5b410f94 sp=0x5b410f80
runtime.goparkunlock(0x4fde30, 0x4a0628, 0xe, 0x14, 0x1)
	/home/zbr/awork/gosrc/src/runtime/proc.go:267 +0x4c fp=0x5b410fb0 sp=0x5b410f94
runtime.runfinq()
	/home/zbr/awork/gosrc/src/runtime/mfinal.go:161 +0x9c fp=0x5b410ff0 sp=0x5b410fb0
runtime.goexit()
	/home/zbr/awork/gosrc/src/runtime/asm_386.s:1629 +0x1 fp=0x5b410ff4 sp=0x5b410ff0
created by runtime.createfing
	/home/zbr/awork/gosrc/src/runtime/mfinal.go:142 +0x62

Application is rather trivial: https://play.golang.org/p/0ZUCNggw7e

Patch for online review:
diff --git a/src/runtime/os_windows.go b/src/runtime/os_windows.go
index 75b8acd..cc001ce 100644
--- a/src/runtime/os_windows.go
+++ b/src/runtime/os_windows.go
@@ -33,6 +33,8 @@ const (
 //go:cgo_import_dynamic runtime._GetThreadContext GetThreadContext%2 "kernel32.dll"
 //go:cgo_import_dynamic runtime._LoadLibraryW LoadLibraryW%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._LoadLibraryA LoadLibraryA%1 "kernel32.dll"
+//go:cgo_import_dynamic runtime._QueryPerformanceCounter QueryPerformanceCounter%1 "kernel32.dll"
+//go:cgo_import_dynamic runtime._QueryPerformanceFrequency QueryPerformanceFrequency%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._ResumeThread ResumeThread%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._SetConsoleCtrlHandler SetConsoleCtrlHandler%2 "kernel32.dll"
 //go:cgo_import_dynamic runtime._SetErrorMode SetErrorMode%1 "kernel32.dll"
@@ -76,6 +78,8 @@ var (
 	_GetThreadContext,
 	_LoadLibraryW,
 	_LoadLibraryA,
+	_QueryPerformanceCounter,
+	_QueryPerformanceFrequency,
 	_ResumeThread,
 	_SetConsoleCtrlHandler,
 	_SetErrorMode,
@@ -290,6 +294,32 @@ func osinit() {
 	// equivalent threads that all do a mix of GUI, IO, computations, etc.
 	// In such context dynamic priority boosting does nothing but harm, so we turn it off.
 	stdcall2(_SetProcessPriorityBoost, currentProcess, 1)
+
+	check_timers()
+}
+
+//go:nosplit
+var systime func(addr uintptr) int64
+var systime_qpc_frequency int64
+
+//go:nosplit
+func check_timers() {
+	systime = systime_memory_mapping
+	t1 := nanotime()
+
+	// wine users will wait for at most 10 usecs at start to detect that wine
+	// doesn't update page containg INTERRUPT_TIME, which means fast time is not supported on wine
+	// we can not simply fallback to Sleep() syscall, since its time is not monotonic,
+	// instead we use QueryPerformanceCounter family of syscalls to implement monotonic timer
+	// https://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx
+	for i := 0; i < 100; i++ {
+		if nanotime() != t1 {
+			return
+		}
+	}
+
+	systime = systime_qpc_syscall
+	stdcall1(_QueryPerformanceFrequency, uintptr(unsafe.Pointer(&systime_qpc_frequency)))
 }

 //go:nosplit
@@ -591,7 +621,7 @@ const (
 )

 //go:nosplit
-func systime(addr uintptr) int64 {
+func systime_memory_mapping(addr uintptr) int64 {
 	timeaddr := (*_KSYSTEM_TIME)(unsafe.Pointer(addr))

 	var t _KSYSTEM_TIME
@@ -607,6 +637,8 @@ func systime(addr uintptr) int64 {
 			osyield()
 		}
 	}
+
+	// this recursively enters systime() call
 	systemstack(func() {
 		throw("interrupt/system time is changing too fast")
 	})
@@ -614,6 +646,18 @@ func systime(addr uintptr) int64 {
 }

 //go:nosplit
+func systime_qpc_syscall(addr uintptr) int64 {
+	// QPC timing is called when memory mapping is not updated, but it still contains the correct initial time
+	// we use this as a base and apply QPC bias
+	base_time := systime_memory_mapping(addr)
+
+	var counter int64
+	stdcall1(_QueryPerformanceCounter, uintptr(unsafe.Pointer(&counter)))
+
+	return base_time + counter * 1000000 / systime_qpc_frequency
+}
+
+//go:nosplit
 func unixnano() int64 {
 	return (systime(_SYSTEM_TIME) -<PHONE_NUMBER>00000000) * 100
 }

@bioothod, we can't accept or review patches on the bug tracker. They have to be in Gerrit.

Again, wine specific code is not ok as the windows builder won't test them.
And I highly doubt we would add a wine builder (I'm pretty sure all.bat
won't pass on wine.)

This is not a request for inclusion, this is a demonstration of the idea @bradfitz
There is no need to test it at build time since it doesn't work yet - it crashes probably because of signal mess, that's why I asked above what could be a reason for error: runtime: stack split at bad time in this code. Testing it in build time is trivial, just call systime_qpc_syscall() directly right after plain systime() has been tested @minux

This is not a request for inclusion, this is a demonstration of the idea

Some won't look at your code for legal reason. For Gerrit access, you must have completed one of the contributor license agreements. Not everything sent via Gerrit is reviewed or submitted.

what could be a reason for fatal error: runtime: stack split at bad time in this code

I am, probably, wrong, but I think you cannot call stdcall1 from systime. stdcall1 function does a lot, and expects runtime to be in a consistent state. But systime (or nanotime) is called from everywhere. We already have osyield and usleep functions that are in a similar situation, perhaps you can do the same in systime.
Alex

Wow, assembler in the high-level language to call a function and save the stack!
I thought it had hidden in the debris like in 90th or something.
Ok, I will dust off my wizard hat, recover the ancient manuscripts and try to cook up a patch.

I wonder how did it work previously, since just 2.5 years ago it was as simple as
int64 runtime·nanotime(void)
 {
    int64 filetime;
    runtime·stdcall(runtime·GetSystemTimeAsFileTime, 1, &filetime)

is there any high-level document describing runtime changes which forced time code to go into hand crafted assembly?

It happens to be much simpler than I thought, no need to switch stacks, problem is in division.
Running 10000000 / 10000000 in NOSPLIT function ends up with panic, since it invokes quite a bit of division functions from runtime.int64div().
By simpler I mean that I do not have to write asm code to invoke QPC syscalls, which for instance requires changing signature of onosstack() to accept uintptr instead of uint32, since otherwise there is no way to get back 64-bit data, with uintptr I can use a pointer at least, although return value is missing.
Code works on i386 if I mark a bunch of division functions in src/runtime/vlrt.go with go://nosplit, what do you think? Something tells me that it may affect runtime by panicing in the middle of the production code if stack has been suddenly exhausted.

For the reference, discussions about stack usage and why some functions had been changed to OS stack:
https://github.com/golang/go/issues/4955
https://codereview.appspot.com/7563043

@bioothod timediv function may help with division.

@dvyukov yeah, timediv() does the trick, thank you
I've made the patch, added QPC test into src/runtime/syscall_windows_test.go, but guys, you force people to use google account with gerrit, and that's a crap, it can not verify newly created account. Either because of russian phone number or whatever else, and sorry, I can not dig into google QA to find out the reason (phone number in screenshot has been changed obviously) @bradfitz @alexbrainman

So I post a patch here, please comment and merge, it works both for i386 and amd64 builds tested on wine, I will run it on real windows machine a bit later.
Patch:
qpc.time.diff.txt
Online:
diff --git a/src/runtime/os_windows.go b/src/runtime/os_windows.go
index 75b8acd..15506da 100644
--- a/src/runtime/os_windows.go
+++ b/src/runtime/os_windows.go
@@ -33,6 +33,8 @@ const (
 //go:cgo_import_dynamic runtime._GetThreadContext GetThreadContext%2 "kernel32.dll"
 //go:cgo_import_dynamic runtime._LoadLibraryW LoadLibraryW%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._LoadLibraryA LoadLibraryA%1 "kernel32.dll"
+//go:cgo_import_dynamic runtime._QueryPerformanceCounter QueryPerformanceCounter%1 "kernel32.dll"
+//go:cgo_import_dynamic runtime._QueryPerformanceFrequency QueryPerformanceFrequency%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._ResumeThread ResumeThread%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._SetConsoleCtrlHandler SetConsoleCtrlHandler%2 "kernel32.dll"
 //go:cgo_import_dynamic runtime._SetErrorMode SetErrorMode%1 "kernel32.dll"
@@ -76,6 +78,8 @@ var (
        _GetThreadContext,
        _LoadLibraryW,
        _LoadLibraryA,
+       _QueryPerformanceCounter,
+       _QueryPerformanceFrequency,
        _ResumeThread,
        _SetConsoleCtrlHandler,
        _SetErrorMode,
@@ -290,6 +294,45 @@ func osinit() {
        // equivalent threads that all do a mix of GUI, IO, computations, etc.
[zbr@odin src]$ cat /tmp/qpc.time.diff.txt
diff --git a/src/runtime/os_windows.go b/src/runtime/os_windows.go
index 75b8acd..15506da 100644
--- a/src/runtime/os_windows.go
+++ b/src/runtime/os_windows.go
@@ -33,6 +33,8 @@ const (
 //go:cgo_import_dynamic runtime._GetThreadContext GetThreadContext%2 "kernel32.dll"
 //go:cgo_import_dynamic runtime._LoadLibraryW LoadLibraryW%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._LoadLibraryA LoadLibraryA%1 "kernel32.dll"
+//go:cgo_import_dynamic runtime._QueryPerformanceCounter QueryPerformanceCounter%1 "kernel32.dll"
+//go:cgo_import_dynamic runtime._QueryPerformanceFrequency QueryPerformanceFrequency%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._ResumeThread ResumeThread%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._SetConsoleCtrlHandler SetConsoleCtrlHandler%2 "kernel32.dll"
 //go:cgo_import_dynamic runtime._SetErrorMode SetErrorMode%1 "kernel32.dll"
@@ -76,6 +78,8 @@ var (
 	_GetThreadContext,
 	_LoadLibraryW,
 	_LoadLibraryA,
+	_QueryPerformanceCounter,
+	_QueryPerformanceFrequency,
 	_ResumeThread,
 	_SetConsoleCtrlHandler,
 	_SetErrorMode,
@@ -290,6 +294,45 @@ func osinit() {
 	// equivalent threads that all do a mix of GUI, IO, computations, etc.
 	// In such context dynamic priority boosting does nothing but harm, so we turn it off.
 	stdcall2(_SetProcessPriorityBoost, currentProcess, 1)
+
+	check_timers()
+}
+
+//go:nosplit
+var systime func(addr uintptr) int64
+var systime_qpc_frequency int32
+var systime_qpc_counter_start int64
+
+//go:nosplit
+func check_timers() {
+	systime = systime_memory_mapping
+
+	var tmp int64
+	stdcall1(_QueryPerformanceFrequency, uintptr(unsafe.Pointer(&tmp)))
+	// this should not overflow, it is a number of ticks of the performance counter per second
+	// panic if overflow
+	if tmp == 0 {
+		panic("QueryPerformanceFrequency syscall returned zero, running on unsupported hardware")
+	}
+	if tmp > (1<<31 - 1) {
+		panic("QueryPerformanceFrequency overflow 32 bit divider, check nosplit discussion to proceed")
+	}
+	systime_qpc_frequency = int32(tmp)
+	stdcall1(_QueryPerformanceCounter, uintptr(unsafe.Pointer(&systime_qpc_counter_start)))
+
+	// wine users will wait for at most 10 usecs at start to detect that wine
+	// doesn't update page containg INTERRUPT_TIME, which means fast time is not supported on wine
+	// we can not simply fallback to Sleep() syscall, since its time is not monotonic,
+	// instead we use QueryPerformanceCounter family of syscalls to implement monotonic timer
+	// https://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx
+	t1 := nanotime()
+	for i := 0; i < 100; i++ {
+		if nanotime() != t1 {
+			return
+		}
+	}
+
+	systime = systime_qpc_syscall
 }

 //go:nosplit
@@ -591,7 +634,7 @@ const (
 )

 //go:nosplit
-func systime(addr uintptr) int64 {
+func systime_memory_mapping(addr uintptr) int64 {
 	timeaddr := (*_KSYSTEM_TIME)(unsafe.Pointer(addr))

 	var t _KSYSTEM_TIME
@@ -607,6 +650,8 @@ func systime(addr uintptr) int64 {
 			osyield()
 		}
 	}
+
+	// NB this recursively enters systime() call
 	systemstack(func() {
 		throw("interrupt/system time is changing too fast")
 	})
@@ -614,6 +659,27 @@ func systime(addr uintptr) int64 {
 }

 //go:nosplit
+func systime_qpc_syscall(addr uintptr) int64 {
+	// QPC timing is called when memory mapping is not updated, but it still contains the correct initial time
+	// we use this as a base and apply QPC bias
+
+	var counter int64 = 0
+	stdcall1(_QueryPerformanceCounter, uintptr(unsafe.Pointer(&counter)))
+
+	return systime_memory_mapping(addr) + int64(timediv((counter - systime_qpc_counter_start) * 10000000, systime_qpc_frequency, nil))
+}
+func SystimeQPCTest() bool {
+	t1 := systime_qpc_syscall(_INTERRUPT_TIME)
+	for i := 0; i < 1000; i++ {
+		t2 := systime_qpc_syscall(_INTERRUPT_TIME)
+		if t2 != t1 {
+			return true
+		}
+	}
+	return false
+}
+
+//go:nosplit
 func unixnano() int64 {
 	return (systime(_SYSTEM_TIME) -<PHONE_NUMBER>00000000) * 100
 }
diff --git a/src/runtime/syscall_windows_test.go b/src/runtime/syscall_windows_test.go
index 11e67df..82dabea 100644
--- a/src/runtime/syscall_windows_test.go
+++ b/src/runtime/syscall_windows_test.go
@@ -933,6 +933,12 @@ func TestLoadLibraryEx(t *testing.T) {
 		have, flags)
 }

+func TimeSystimeTest(t *testing.T) {
+	if !runtime.SystimeQPCTest() {
+		t.Fatal("runtime.systime_qpc_syscall() has stalled")
+	}
+}
+
 var (
 	modwinmm    = syscall.NewLazyDLL("winmm.dll")
 	modkernel32 = syscall.NewLazyDLL("kernel32.dll")

@bioothod, sorry, for legal reasons we can't accept a patch here on the bug tracker.

OMG, are you serious? You do allow proprietary binary crap in the executable and can not accept several lines diff which fixes real bug basically rendering system unusable in the most popular crossplatform environment on the world without shitty bureaucracy, which involves registering new google account, which virtually refuses to verify new user (probably because this phone number is already used in a completely irrelevant setup)?
Guys, are you aware of the 21 century auth technologies? OAuth maybe, did you hear this word? Do you know about github? Maybe words 'signed-off-by' highlight something - those beardy guys in linux-kernel use it. Or maybe code of conduct, facebook requires to sign a special to accept patches...
What's wrong with you?
Someone, please copy this patch into review system or rewrite it the way you like, I've shown the basic idea, implementation fixes time bug in wine environment and works without overhead for common windows users.

 Where are the proprietary binary blobs in Go?

it can not verify newly created account. Either because of russian phone number or whatever else, and sorry, I can not dig into google QA to find out the reason (phone number in screenshot has been changed obviously) @bradfitz @alexbrainman

I do not work for Google, and have no special contacts there. Sorry.
Alex

@minux I talked about binaries distributed via vendoring feature

@bioothod, I'm sorry you're upset, but we can't make legal exceptions just because you're upset.
The bureaucracy has pros and cons. You've hit a con. The pros are that we have a very clean contribution history and legal story. Users of Go appreciate that.
If you manage to create a Google account we can start a code review for you. Until then, we'll just have to wait.

@alexbrainman I'm not insisting on fixing google account bugs of course, I'm just pointing that code review process is flawed at best imo, and I would describe it in more colourful details actually...
I've shown the idea of how to fix it, I would gladly submit it for review and fix this bug, but I believe we have a show stopper so far, I will try to resolve it, but hardly see what can be done except finding another phone number.
If you think that this implementation is correct, consider pushing it (rewritten, rethought or whatever else) upstream

On Sun, Jan 15, 2017 at 7:16 PM, Evgeniy Polyakov<EMAIL_ADDRESS>wrote:

@minux https://github.com/minux I talked about binaries distributed via
vendoring feature

The vendoring feature is about source codes.
Go does have support for binary only packages (//go:binary-only-package),
but we intentionally makes it impossible to distribute them using regular
"go get". And there is a little-known feature of *.syso files, but there is
no proprietary binary syso blobs in the Go repos.

@bradfitz flawed google account requirement has virtually zero correlation with go users who appreciate clean contribution. Your words imply that facebook or linux-kernel contribution history and legal story are not clean enough, are you sure that requiring flawed google account is strong enough for such words?

@bioothod, I'm not going to debate our CLA requirements in this forum. If you're looking for an argument, look elsewhere.

@minux I didn't tell about golang repos per se, but legal greyarea of the binary packages combined with vendoring feature. This was a reply to claims that bugfix like about is not clear enough from legal point of view. But that's irrelevant you are correct.

@bradfitz I'm curious, whether I need to have posts in google+ social network and how many, apparently this might be a requirement too like using google account.
Apart from the size of google drive had to be used to review the patch, is it ok from technical point of view? I.e. does it break something, or highlight some issue with stalled time detection, or maybe incorrect syscall usage?

Updated QPC fallback time implementation. On the real hardware 100 loop iterations are actually either too quick (which is unlikely), it windows updates those mappings on some other events which are not happening in go.
Anyway, running a following loop on the real hardware ended up with unchanged times:
+       t1 := nanotime()
+       for i := 0; i < 100; i++ {
+               if nanotime() != t1 {
+                       return
+               }
+       }

so I updated test to actually perform a sleep, I call usleep(2), which can take a while, it is 1-2 ms on wine. What do you think, is it an acceptable delay or not?
If it is not, there is plan B.
Interrupt time is a time since boot, and I can not tell whether this memory mapping is ever updated on wine, but at least at osinit() time it is zero (which sounds logically since process has just booted)  and can be used to check whether timing calls should be updated or not.
Embedded patch for review. If you are still afraid of letters and believe that commenting patch on github causes legal issues, headache and baldness, @noxiouz will push it into gerrit on my behalf
diff --git a/src/runtime/os_windows.go b/src/runtime/os_windows.go
index 75b8acd..6e34b04 100644
--- a/src/runtime/os_windows.go
+++ b/src/runtime/os_windows.go
@@ -33,6 +33,8 @@ const (
 //go:cgo_import_dynamic runtime._GetThreadContext GetThreadContext%2 "kernel32.dll"
 //go:cgo_import_dynamic runtime._LoadLibraryW LoadLibraryW%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._LoadLibraryA LoadLibraryA%1 "kernel32.dll"
+//go:cgo_import_dynamic runtime._QueryPerformanceCounter QueryPerformanceCounter%1 "kernel32.dll"
+//go:cgo_import_dynamic runtime._QueryPerformanceFrequency QueryPerformanceFrequency%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._ResumeThread ResumeThread%1 "kernel32.dll"
 //go:cgo_import_dynamic runtime._SetConsoleCtrlHandler SetConsoleCtrlHandler%2 "kernel32.dll"
 //go:cgo_import_dynamic runtime._SetErrorMode SetErrorMode%1 "kernel32.dll"
@@ -76,6 +78,8 @@ var (
 	_GetThreadContext,
 	_LoadLibraryW,
 	_LoadLibraryA,
+	_QueryPerformanceCounter,
+	_QueryPerformanceFrequency,
 	_ResumeThread,
 	_SetConsoleCtrlHandler,
 	_SetErrorMode,
@@ -290,6 +294,68 @@ func osinit() {
 	// equivalent threads that all do a mix of GUI, IO, computations, etc.
 	// In such context dynamic priority boosting does nothing but harm, so we turn it off.
 	stdcall2(_SetProcessPriorityBoost, currentProcess, 1)
+
+	check_timers()
+}
+
+//go:nosplit
+var _unixnano func() int64
+//go:nosplit
+var _nanotime func() int64
+
+//go:nosplit
+func unixnano() int64 {
+	return _unixnano()
+}
+
+//go:nosplit
+func nanotime() int64 {
+	return _nanotime()
+}
+
+var systime_qpc_frequency int32
+var systime_qpc_counter_start int64
+var qpc_base_nano_system, qpc_base_nano_interrupt int64
+
+//go:nosplit
+func check_timers() {
+	_unixnano = unixnano_mm
+	_nanotime = nanotime_mm
+
+	// wine will actually have a zero nanotime, since it is a time since boot, and starting the process is actually a boot
+	if unixnano() != 0 && nanotime() != 0 {
+		// users will wait for 2 usecs at start to detect that system doesn't update page containg INTERRUPT_TIME,
+		// which means fast time is not supported on wine
+		// we can not simply fallback to Sleep() syscall, since its time is not monotonic,
+		// instead we use QueryPerformanceCounter family of syscalls to implement monotonic timer
+		// https://msdn.microsoft.com/en-us/library/windows/desktop/dn553408(v=vs.85).aspx
+		t1 := nanotime()
+		usleep(2)
+		t2 := nanotime()
+		if t1 != t2 {
+			return
+		}
+	}
+
+	qpc_base_nano_system = unixnano_mm()
+	qpc_base_nano_interrupt = nanotime_mm()
+
+	_unixnano = unixnano_qpc
+	_nanotime = nanotime_qpc
+
+	var tmp int64
+	stdcall1(_QueryPerformanceFrequency, uintptr(unsafe.Pointer(&tmp)))
+	// this should not overflow, it is a number of ticks of the performance counter per second,
+	// its resolution is at most 10 per usecond (on wine, even smaller on real hardware), so it will be at most 10 millions here,
+	// panic if overflows
+	if tmp == 0 {
+		panic("QueryPerformanceFrequency syscall returned zero, running on unsupported hardware")
+	}
+	if tmp > (1<<31 - 1) {
+		panic("QueryPerformanceFrequency overflow 32 bit divider, check nosplit discussion to proceed")
+	}
+	systime_qpc_frequency = int32(tmp)
+	stdcall1(_QueryPerformanceCounter, uintptr(unsafe.Pointer(&systime_qpc_counter_start)))
 }

 //go:nosplit
@@ -607,6 +673,8 @@ func systime(addr uintptr) int64 {
 			osyield()
 		}
 	}
+
+	// NB this recursively enters systime() call
 	systemstack(func() {
 		throw("interrupt/system time is changing too fast")
 	})
@@ -614,15 +682,44 @@ func systime(addr uintptr) int64 {
 }

 //go:nosplit
-func unixnano() int64 {
+func unixnano_mm() int64 {
 	return (systime(_SYSTEM_TIME) -<PHONE_NUMBER>00000000) * 100
 }

 //go:nosplit
-func nanotime() int64 {
+func nanotime_mm() int64 {
 	return systime(_INTERRUPT_TIME) * 100
 }

+//go:nosplit
+func systime_qpc() int64 {
+	var counter int64 = 0
+	stdcall1(_QueryPerformanceCounter, uintptr(unsafe.Pointer(&counter)))
+
+	// returns number of nanoseconds
+	return int64(timediv((counter - systime_qpc_counter_start) *<PHONE_NUMBER>, systime_qpc_frequency, nil))
+}
+
+//go:nosplit
+func unixnano_qpc() int64 {
+	// returns number of nanoseconds since Unix Epoch
+	return qpc_base_nano_system + systime_qpc()
+}
+
+//go:nosplit
+func nanotime_qpc() int64 {
+	// returns number of nanoseconds since boot
+	return qpc_base_nano_interrupt + systime_qpc()
+}
+
+func SystimeQPCTest() bool {
+	t1 := unixnano_qpc()
+	usleep(5)
+	t2 := unixnano_qpc()
+	return t1 != t2
+}
+
+
 // Calling stdcall on os stack.
 // May run during STW, so write barriers are not allowed.
 //go:nowritebarrier
diff --git a/src/runtime/syscall_windows_test.go b/src/runtime/syscall_windows_test.go
index 11e67df..82dabea 100644
--- a/src/runtime/syscall_windows_test.go
+++ b/src/runtime/syscall_windows_test.go
@@ -933,6 +933,12 @@ func TestLoadLibraryEx(t *testing.T) {
 		have, flags)
 }

+func TimeSystimeTest(t *testing.T) {
+	if !runtime.SystimeQPCTest() {
+		t.Fatal("runtime.systime_qpc_syscall() has stalled")
+	}
+}
+
 var (
 	modwinmm    = syscall.NewLazyDLL("winmm.dll")
 	modkernel32 = syscall.NewLazyDLL("kernel32.dll")

Embedded patch for review.

Again, please stop putting patches here. Nobody is allowed to review this code unless it comes via Gerrit.
If I see any further patches or any code reviews in this thread I will unfortunately be forced to lock this thread.

@bioothod, I'm sorry you're upset, but we can't make legal exceptions just because you're upset.

If laywers dictate how you accept patches, well, one day they will come after you. Run away from the project before they do.

I updated test to actually perform a sleep, I call usleep(2), which can take a while, it is 1-2 ms on wine. What do you think, is it an acceptable delay or not?

I think we want to do as little as possible on startup. What happened to "searching for wine_get_version function in ntdll.dll" idea? We already search ntdll.dll for other things, so you would need to add just a couple of lines.

If you are still afraid of letters and believe that commenting patch on github causes legal issues, headache and baldness

I have been bold for years. And have no problems with headache. But I wouldn't want legal troubles for the Go Team. I think we all want them to be happy and write code and do code review and ... instead of wasting their time with legal matters. So if @noxiouz wants to send this change, that would be good. I would do it myself, but my TODO list is very long, and you would have to wait awhile.
Alex

Patch has been submitted into gerrit, please comment (preferably here) and merge
https://go-review.googlesource.com/#/c/35710/

Kudos to Anton Tiurin @noxiouz for this

Kudos to Anton Tiurin @noxiouz for this

You should note that https://go-review.googlesource.com/#/c/35710/ will not progress (Brad set -2 there) until CLA is cleared.
Alex


Enforcing SSL

There was a discussion a little while back in regards to this feature, and since I never got around to implementing it, I thought I would throw it here until somebody finds the time..
The idea was to have a controller extension defining an "enforce SSL" flag, where a request filter is used to determine whether SSL should be enforced for the specific page.

I'm going to close this for now, as I believe the entire site should be enforced to SSL as opposed to a single page.


sankey-panel plugin is not working with grafana 7.4.3

Do you have a suggested grafana version which can use all the suggested plugins?? There is issue with saney-panel plugin as mentioned https://github.com/kumaravel29/grafana-sankey-panel/issues/31

Hello,
Sorry to answer late :/. So busy.
I got the same yes 0.0. I will try to check later.

This is a sankey issue, not really the dashboard.  But they do mention the resolution both in the issue you referenced and on the main sankey page.
In this plugin, the Sankey daigram panel has been used to visualize the distribution between 2 keys. Important - This panel is dependent on the google chart and requires loader.js which needs to be included in the Grafana index.html file
Copy the script line below into:   /usr/share/grafana/public/views/index.html


Distributed DNS Health Checks

[ ] https://github.com/Kuadrant/kuadrant-dns-operator/issues/15

@pehala who is a good person to work with @philbrookes on this epic?

Closing this since we don't intend, at the moment, to progress this instead focusing on AWS provider health checks https://github.com/Kuadrant/dns-operator/issues/44


docs: Fix missing a code block end quote

Fix a markdown code block quoting problem.
Found at reading the document

Coverage remained the same at 94.65% when pulling 6f1a42ae4419213659993b4c1d6efba7f904644b on zhangciwu:master into 063e19ff4b47c09b6c660c591b89079c816ae5dd on dsherret:master.

Thanks! 😄
I don't pay enough attention to the markdown preview to catch these mistakes.


Fix some container commands

Looks like the same command is used for multiple sidecar containers. This change is supposed to fix the process run within the sidecar containers.

@cschwede let me know if I'm not parsing it properly, but I assumed that was leftover

@cschwede let me know if I'm not parsing it properly, but I assumed that was leftover

Thanks @fmount, good catch! I'll add some updates to the kuttl tests in a follow up to avoid regressions in the future.

/retest

looks like there are issues w/ prow jobs atm, we can recheck the failing test later I guess

/retest


added gifs

Added a couple more phrases and GIF to be displayed from GIPHY API.

improved giphy search and will request that be pulled instead


Using Context Files Does Not Seem To Work

I am running a local model from Ollama called qwen2:latest.  It is running locally via Ollama and on other GUI front ends I can interact with it normally. It interacts normally in SystemSculpt as long as I'm not asking it about local files.
I put a file in the Context Files box. Maybe I'm dense but as far as I can tell according to the SystemSculpt documentation, that's all I have to do to get the model to take that note into context, right?
So I ask the model to summarize the document in five bullet points. This is what I get in return:
To provide a summary using a linked file, I would need the actual content or data from the file to create a textual summary with five bullet points. However, since I don't have direct access to external files or links in this interaction, I can guide you on how to manually create a summary:
As far as I can tell, it has absolutely no awareness of the file sitting in the Context Files box.
Is there something I can do to debug this?

OK, I think I tried it with a couple local models, but I don't think I tried it with an online model. I'll test it out maybe today. I wouldn't be surprised if it was an issue with local models.

I've discovered the problem. The command doesn't work if the context file is already the currently opened file. It has to be explicitly opened using the link to the file.
I would suggest that an improvement would be to automatically open the file when a linked file is added to the Context Files modal. Better yet, open the file internally in the context to be sent with the prompt so the file doesn't have to be opened in a tab. Even better yet, allow the user to edit the internally opened file before sending to add additional context to the file. Also allow any and all currently opened file(s) to be referenced as part of the context. The BHO plugin does this for one file.


[dart2wasm] Argument types are not checked in dynamic invocations

This causes a few test failures and I was able to come up with two tiny repros, one causes Wasm trap, one crashes in compile time.
This one fails with Wasm trap:
class B {
  void aaa(double x) {}
}

void main() {
  (B() as dynamic).aaa(123);
}

wasm-function[722]:0x22959: RuntimeError: illegal cast
RuntimeError: illegal cast
    at aaa dynamic trampoline (wasm://wasm/000aa906:wasm-function[722]:0x22959)
    at main (wasm://wasm/000aa906:wasm-function[50]:0x15fa9)
    at $main (wasm://wasm/000aa906:wasm-function[51]:0x15fb0)
    at Module.invoke (/Users/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/dart2wasm_runtime.mjs:410:28)
    at main (pkg/dart2wasm/bin/run_wasm.js:46:21)

If I add an optional parameter to the method then it crashes in compile time:
class B {
  void aaa(double x, [int? start]) {}
}

void main() {
  (B() as dynamic).aaa(123);
}

Exception in IntJudgment at file:///Users/omersa/dart/sdk/sdk/test.dart:18:24
Unhandled exception:
type 'NumType' is not a subtype of type 'RefType' in type cast
#0      Translator.convertType (package:dart2wasm/translator.dart:908:28)
#1      CodeGenerator.wrap (package:dart2wasm/code_generator.dart:622:18)
#2      CodeGenerator.visitArgumentsLists (package:dart2wasm/code_generator.dart:2276:7)
#3      DynamicDispatcher._callDynamic.<anonymous closure> (package:dart2wasm/dynamic_dispatch.dart:84:19)
#4      DynamicDispatcher._emitDynamicCallBody (package:dart2wasm/dynamic_dispatch.dart:182:22)
#5      DynamicDispatcher._callDynamic (package:dart2wasm/dynamic_dispatch.dart:60:26)
#6      DynamicDispatcher.emitDynamicCall (package:dart2wasm/dynamic_dispatch.dart:480:16)
#7      CodeGenerator.visitDynamicInvocation (package:dart2wasm/code_generator.dart:1499:32)
#8      DynamicInvocation.accept1 (package:kernel/ast.dart:5555:9)
#9      CodeGenerator.wrap (package:dart2wasm/code_generator.dart:621:37)
#10     CodeGenerator.visitExpressionStatement (package:dart2wasm/code_generator.dart:965:5)
#11     ExpressionStatement.accept (package:kernel/ast.dart:9303:43)
#12     CodeGenerator.visitStatement (package:dart2wasm/code_generator.dart:632:12)
#13     CodeGenerator.visitBlock (package:dart2wasm/code_generator.dart:731:7)
#14     Block.accept (package:kernel/ast.dart:9359:43)
#15     CodeGenerator.visitStatement (package:dart2wasm/code_generator.dart:632:12)
#16     CodeGenerator.generateBody (package:dart2wasm/code_generator.dart:437:7)
#17     CodeGenerator.generate (package:dart2wasm/code_generator.dart:186:12)
#18     Translator.translate (package:dart2wasm/translator.dart:459:15)
#19     compileToModule (package:dart2wasm/compile.dart:86:21)
<asynchronous suspension>
#20     main (file:///Users/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/dart2wasm.dart:120:23)

Another repro:
void test(l1) {
  l1.insert(null, 5);
}

void main() {
  test([0, 1, 2, 3, 4]);
}

AOT output:
Unhandled exception:
type 'Null' is not a subtype of type 'int' of 'index'
#0      testModifiableList (file:///usr/local/google/home/omersa/dart/sdk/sdk/test.dart:2:6)
#1      main (file:///usr/local/google/home/omersa/dart/sdk/sdk/test.dart:6:3)
#2      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#3      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

dart2wasm traps:
wasm-function[728]:0x22baa: RuntimeError: dereferencing a null pointer
RuntimeError: dereferencing a null pointer
    at insert dynamic trampoline (wasm://wasm/000ac746:wasm-function[728]:0x22baa)
    at testModifiableList (wasm://wasm/000ac746:wasm-function[727]:0x22b25)
    at main (wasm://wasm/000ac746:wasm-function[53]:0x160ff)
    at main tear-off (wasm://wasm/000ac746:wasm-function[54]:0x16105)
    at _invokeMain (wasm://wasm/000ac746:wasm-function[64]:0x163d7)
    at Module.invoke (/usr/local/google/home/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/dart2wasm_runtime.mjs:410:28)
    at main (/usr/local/google/home/omersa/dart/sdk/sdk/pkg/dart2wasm/bin/run_wasm.js:46:21)

My plan to fix this:

For every member that can be called in a dynamic invocation we will create a new Reference that checks the argument types and calls the actual method.

These references need to be in the dispatch table, because every override of a method will need a different type checking code. For example:
class A {}
class B extends A {}

class C1 {
  void f(B b) {}
}

class C2 extends C1 {
  @override
  void f(A a) {}
}

void main() {
  // Runtime type check fails
  dynamic x = C1();
  x.f(A());

  // Runtime type check passes
  dynamic y = C2();
  y.f(A());
}

So the selector for f will have different dynamic invocation targets for each override of f, and in the invocation site we select the right one similar to virtual calls: dispatchTable[classId + dynamicInvocationOffset].

Mapping a member to the dynamicInvocationSelectorOffset used above is already handled by DispatchTable.selectorsForDynamicNode.

When populating the dynamicGets, dynamicSets, and dynamicInvocations maps (used by DispatchTable.selectorsForDynamicNode), we create a new reference, instead of using the existing reference for the non-dynamic invocation of the member.

When generating code for a reference (CodeGenerator.generate) we need to check if the reference is for a dynamic invocation target so that we can generate type checks.
This is handled similarly to how we check for tear-off references: we add a new expando in reference_extensions.dart with accessors. The expando will be populated as we create new references for dynamic invocations in DispatchTable.build.

The code for a dynamic invocation target does the type checks and calls the instance invocation target.
Generating type checks should be straightforward as we do it in other places in the code generator.

When generating the code for a dynamic invocation we already know the class ID of the receiver and the instance invocation target's table offset. So after the type check we just generate a call_indirect with a constant as offset.
(I think we should be able to generate a direct call here?)

I think that's it. Any feedback before I start implementing this @askeksa-google @joshualitt?

If you are implementing this as part of the existing dynamic call mechanism, then we should be able to always emit a direct call, so it's not necessary to put the type checking trampoline into the dispatch table.
The existing dynamic call code already has a concept of emitting trampolines to share code between calls of the same selector in most cases. Would it be possible to reuse that mechanism for the type checking trampolines (i.e. do the type check inside the existing trampoline) instead of adding a separate kind of reference? In cases where we can't use a trampoline, the checks can be performed inline in the call like the existing checks. Basically this amounts to just adding the type checks to the call code wherever that is emitted.

Would it be possible to reuse that mechanism for the type checking trampolines (i.e. do the type check inside the existing trampoline) instead of adding a separate kind of reference? In cases where we can't use a trampoline, the checks can be performed inline in the call like the existing checks. Basically this amounts to just adding the type checks to the call code wherever that is emitted.

Hm, right, I think this should be possible and it'll be simpler too. I'll implement this.
I was assuming we want to avoid duplicating type checking code every time a member is called in dynamic invocation (emitTypeTest looks scary), but maybe that's OK.

For the type checks I was hoping to use makeType to push the _Type object of a member parameter onto the stack, to be passed to _isSubtype, but that function wants to resolve type parameters using the current class or function's type arguments. In dynamic invocations we want to resolve type parameters using the receiver's type arguments. For example:

class C<A> {
  A _a;

  C(this._a);

  void setA(A a) {
    _a = a;
  }
}

void main() {
  dynamic a = C<int>(123);
  a.setA(123);
}

In a.setA(...), A needs to be resolved using a's type arguments. makeType wants to resolve it using current class (which doesn't exist in this example since the call site is not in a class) or current function's (main) type arguments.
To allow specifying how a type parameter should be resolved in makeType I refactored it and added a parameter:
w.ValueType makeType(CodeGenerator codeGen, DartType type,
    void instantiateTypeParameter(TypeParameter parameter)) { ... }

This works fine but reveals another problem: makeType generates code in the current function being compiled in CodeGenerator, but when generating a trampoline we create a new function and generate instructions in that function. CodeGenerator cannot be used in when generating code in a trampoline function.
So we need to decouple makeType from CodeGenerator if we want to use it when generating trampolines. We could still have a CodeGenerator.makeType method that works as before, calling the more general makeType that generates in a given function.

I think your idea of generating type testing stubs via a special kind of reference would solve this. Then the generation of that stub can set up the proper environment for this and the type arguments in the code generator like it's done for ordinary bodies, async wrappes and lambdas. It would still not need to go in the dispatch table, since the calls from the dynamic calls/trampolines can be direct.
This is somewhat similar to how the VM does it, actually. It also has special stubs in front of methods to type check arguments for dynamic calls.

I think your idea of generating type testing stubs via a special kind of reference would solve this ...

Good point.
I tried decoupling makeType and CodeGenerator. The main difficulty seems to be this code when creating a function type: https://github.com/dart-lang/sdk/blob/e970e6c8256caecafb7b7d0c52ae8de8999429aa/pkg/dart2wasm/lib/types.dart#L408-L423
This allocates parameter list for the function type, and the list elements are compiled from expressions. So makeType needs to be able to compile expressions.
I think it would probably be easier to refactor CodeGenerator to allow generating code in a different DefinedFunction than the initial one. State like break labels, finalizers, mapping from function type parameters to Wasm locals etc. need to be saved and restored when overriding the current function. I think it should be doable.
Before doing that, it should simplify the code a little bit it we split CodeGenerator into multiple classes for compiling expressions, statements, and initializers. With this we will need a class for the "current function state" (since they will generate code in the same function and need that state) with labels, finalizers etc., and once we have that it should be simple to save and restore that state when generting code in a different function (when generating trampolines).

I think it would probably be easier to refactor CodeGenerator to allow generating code in a different DefinedFunction than the initial one

I just realized it's actually very easy to create a new CodeGenerator. I will try creating a new CodeGenerator in _emitTrampoline and generate code using that CodeGenerator in makeType.

It could make sense to refactor some of the code generator at some point, yes. Is any of that necessary with the type checking stub approach, though?

I just realized it's actually very easy to create a new CodeGenerator. I will try creating a new CodeGenerator in _emitTrampoline and generate code using that CodeGenerator in makeType.

This works and it takes only a few lines, but the refactoring I mentioned above in makeType to take a "instantiate type parameter" argument:
w.ValueType makeType(CodeGenerator codeGen, DartType type,
    void instantiateTypeParameter(TypeParameter parameter)) { ... }

is a bit difficult to do because makeType is sometimes called recursively with unrelated code generator functions like "wrap" between the recursive calls. So instantiateTypeParameter argument needs to be passed to most (if not all) code generator methods, or needs to be made a field.

At the bottom _emitTrampoline passes the right instantiateTypeParameter argument, but code generator then calls makeType again with the original initialTypeParameter that resolves the parameter using this. In the middle there's wrap and a visitor method, which need to be aware of instantiateTypeParameter argument and pass it along.
We could try making instantiateTypeParameter a field, but trying to override it feels hacky either way.
Maybe the stub approach is the best.

I realized as I read the code a bit more that generating type-checking wrapper functions (AOT calls these "dynamic invocation forwarders") is actually much simpler than my original plan above.
First, (not quite a blocker for this issue but a simplification) we don't need dispatch table or selectors to compile dynamic invocations. We should decouple dispatch table and dynamic invocation code. I will do that first.
(I suspect the reason why we used dispatch table was because we thought dynamic calls need to go through the dispatch table in runtime, but that's not the case. I'm updating indirect calls via the dispatch table with direct calls in https://dart-review.googlesource.com/c/sdk/+/271322. Another CL in progress to remove rest of the uses of selectors and dispatch table.)
After that, we add a pass before code generation for members to create new references for dynamic invocation forwarders for members, using the methodOrSetterCalledDynamically metadata. Mapping members to forwarders and checking if a member being compiled is a forwarder will be done the way described in my original plan (with an expando).
The rest should be trivial. When compiling a forwarder we generate type checks using makeType and _isSubType. I have this part working in https://dart-review.googlesource.com/c/sdk/+/270641 and then generate a direct call to the member. (reference.asMember.enclosingClass should give the target class)

Sounds like a good plan! A few thoughts...

After that, we add a pass before code generation for members to create new references for dynamic invocation forwarders for members, using the methodOrSetterCalledDynamically metadata.

I would expect that a separate pass is not needed here. The stubs can be requested lazily by the dynamic call generation by passing the dynamic stub reference to functions.getFunction. I think this will work better together with tree shaking.

The rest should be trivial. When compiling a forwarder we generate type checks using makeType and _isSubType. I have this part working in https://dart-review.googlesource.com/c/sdk/+/270641 and then generate a direct call to the member. (reference.asMember.enclosingClass should give the target class)

Use emitTypeTest for the type tests. This will make sure that future optimizations to type testing (which will be handled by emitTypeTest) will also apply to these dynamic checks.

I would expect that a separate pass is not needed here. The stubs can be requested lazily by the dynamic call generation by passing the dynamic stub reference to functions.getFunction. I think this will work better together with tree shaking.

We discussed this offline but just to document it here, the pass is to map a member name to the forwarder function that will check the class ID, then check the arguments type, and then call the actual member.

In https://github.com/osa1/sdk/tree/dart2wasm_forwarders I have a prototype that again blocked by makeType (or other functions that use makeType, like emitTypeTest).
In that branch I wanted to avoid adding a new pass and I kept using DispatchTable.build to collect members that are dynamically invoked.
For each member that can be dynamically accessed (in a dynamic set, get, or invocation) I generate a function that checks receiver class ID with the target class IDs, and when a match is found I generate the shape check (number of arguments must match) and then type checks for the arguments before calling the member.
The problem is these forwarder functions are not class members, so CodeGenerator cannot be used during code generation for those, and as a result I can't emit type checks for the reasons described in earlier comments.
So what we really want to do is, we still want functions for dynamic invocations of members, but type checks need to be done by instance methods of the receivers, which will have access to this, and know about class-bound type parameters.
For example, suppose we have this class:
class C<A> {
  void f<B>(A a, B b) { ... }
}

and this dynamic invocation:
(C<int>() as dynamic).f<bool>(true, "test")

The type tests will compare the argument type A with bool, and B with String. For that we need to be able to resolve A and B. The easiest way to do is to call a member of the same receiver (the receiver in the dynamic invocation) which does the type checking. That way we can emit a type test in the method body with emitTypeTest and it will be able to resolve A.
For B, I think we somehow need to adjust CodeGenerator.typeLocals so that makeType (or emitTypteTest etc.) will resolve B to the actual type argument. I'm not 100% sure on how to do this. Basically we need to adjust CodeGenerator state so that instantiateTypeParameter will resolve B to the actual type argument.
Where to compare the call site and member shapes? When we find the class matching the receiver's class ID, we need to check that the member and the call sites have compatible "shapes", i.e. it should have required positional parameters, should either have no type args or the same number of type args as the member, it should have required named arguments and no extra named arguments.
This check can be done in the forwarder or in the type-checking members. The code should not be too complicated, so we won't need CodeGenerator. Any complicated parts can be implemented in Dart.
Note that if we implement shape checking in the forwarder, then the type-checking methods can take type arguments, instead of taking a Wasm array of types representing the type arguments, but this probably won't simplify the type checks. In the example above, B is bound in the member f's type, and we won't have the binder in the body of the type-checking method even if it takes a type argument.
So for now I'm thinking of implementing simple forwarders and check everything (shape and types) in instance members.

I almost finished implementing the plan above in https://github.com/osa1/sdk/tree/dart2wasm_forwarders_2. Type checking of named arguments implemented as well. However type checking with type parameters do not work as expected yet. It needs a bit more debugging..

Type checking works nicely as well. I just need to implement dynamic sets and then we can start testing and refactoring.

WIP in https://dart-review.googlesource.com/c/sdk/+/272261.


Added functionality for blogs to be part of a series

Fixes #89
Description
( i have created 2 series and added random post to be part of that series, i have reverted change during PR).
Added the functionality for the blog to be part of series, and there is a separate page for listing all the series and in blog listing it is displayed that a particular blog is a part of particular series.
Other information
In the edit section in the admin area, there is a drop-down from where one can include a particular blog in particular series.
Proof of work
(All the changes like including a blog in series have been reverted while creating PR and i will update the documentation on how to add this functionality ).

Listing all the series

Checking a particular series and all the blogs which are part of it

Displaying inside each post if it is part of any series

On blog list page, showing if blog is part of any series

-Editing option in admin area for including a blog in a series(3rd option)

Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the master branch of the repository.
[x] My commit messages follow best practices.
[x] I added tests for the changes I made (if applicable).
[ ] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no
visible errors.

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

Just acknowledging that I've seen this PR, I'll try and review it tomorrow.

@kgodey ok mam, and to add a series go in contents directory and inside series directory create a new directory with the name you prefer and inside that directory add contents.lr file with the given content.
name: name-of-series
And after this this series will be available in drop-down in post edit section in admin area.
[ I will update these instructions once you will verify].
Thank you

@kgodey ok mam, i will do as you have suggested.
Thank you.

@kgodey mam, I have changed accordingly, please have a look.
Thank you.

@kgodey hello mam, i have updated the changes.
Top part of the blog

Listing all post in a series

All series

Looks good. I'd like a couple of other style changes but I'll make them once this is merged.

Here's the PR changing the style and adding series information to existing posts: https://github.com/creativecommons/creativecommons.github.io-source/pull/153

Thank you for working on this, @techievivek. :)

@kgodey , Thank you mam for allowing me to work on this. I learnt a lot during this and got to know codebase better.
I would like to move further and work on other products of CC, while continue fixing minor bugs in this repo.
Thank you mam.

@kgodey, mam why that series is not visible on site?

@kgodey i got it mam, it is still under review.

@techievivek you're very welcome. we'd love to have you contribute more.

@kgodey thank you, mam.


Dask Dashboard Inaccessible without Sidecar

If you set up Dask without using the sidecar, i.e., through:
cluster = CoffeaCasaCluster()
cluster.scale(jobs=30)
client = Client(cluster)

Then it starts up fine and sends the message:
distributed.scheduler - INFO - Clear task state
distributed.scheduler - INFO -   Scheduler at: tls://<IP_ADDRESS>:8786
distributed.scheduler - INFO -   dashboard at:                     :8787
distributed.scheduler - INFO - Receive client connection: Client-a10a7331-716e-11eb-8a6a-fafc12dcbab4
distributed.core - INFO - Starting established connection

However, the link it gives to the Dask scheduler is inaccessible. This seems bad - the dashboard gives a lot of useful info about what happens during the analysis run. Is this intentional because of some issues it'd make with the sidecar scheduler, or can it be fixed?

It seems it is working in current version:

@mat-adamec could you please recheck whenever you will have time and close this issue?

The buttons in the sidebar work, but the link to the dashboard itself doesn't.
I think this is okay. The buttons have all the same information. But it isn't necessarily intuitive to people who have used Dask with the regular scheduler dashboard before. I think this would be a good thing to put into common issues.


Update dot net core

Update DotNet Core SDK and Runtime to the latest version on build agents
Windows-2019
Ubuntu 18.04.4

Is there an issue associated with this PR? Why are we adding 3.1 as a new baseline? We should  consider moving our baseline up to 3.1 if we want a newer version otherwise we start to pay a higher cost for running multiple versions.

Is there an issue associated with this PR? Why are we adding 3.1 as a new baseline? We should consider moving our baseline up to 3.1 if we want a newer version otherwise we start to pay a higher cost for running multiple versions.

Yes, there is Issue 11014.
I spoke with @AlexGhiondea  he said we should run both netcoreapp2.1 and netcoreapp3.1 for a few months before finally switching to only 3.1.

I spoke with @AlexGhiondea he said we should run both netcoreapp2.1 and netcoreapp3.1 for a few months before finally switching to only 3.1.

@AlexGhiondea are you suggesting we move the supported baseline of .NET Core up to 3.1 in a few months?

Closing this, will pick up the work later.


About training time

Thanks for your nice work! Compared with the concurrent 4D Gaussian paper, your results are better and the rendering speed is faster. I am more concerned about the training time, but I did not find this information in the paper.

The training time is highly influenced by the chosen hyperparameters such as the number of the initial points, schedule of densification/pruning, batch size, the chosen of regularization, and etc.
Loss computation as well as the dataloader contributes a large portion of the training time, which can be further optimized with the improvement of the training framework.
For example, training 30K iters on the hellwarrior took 18min on a RTX 4090 Laptop GPU.
Actually, considering that the computation of SSIM loss constitutes the vast majority of time in each training iteration, the time cost per training iteration for the recent GS-based approaches should exhibit minimal differences. In contrast, the rendering speed is more representative of the efficiency of the proposed scene representation itself.

The training time is highly influenced by the chosen hyperparameters such as the number of the initial points, schedule of densification/pruning, batch size, the chosen of regularization, and etc.
Loss computation as well as the dataloader contributes a large portion of the training time, which can be further optimized with the improvement of the training framework. For example, training 30K iters on the hellwarrior took 18min on a RTX 4090 Laptop GPU.
Actually, considering that the computation of SSIM loss constitutes the vast majority of time in each training iteration, the time cost per training iteration for the recent GS-based approaches should exhibit minimal differences. In contrast, the rendering speed is more representative of the efficiency of the proposed scene representation itself.

i see, thanks for your reply!


Replace setupTestFrameworkScriptFile with setupFilesAfterEnv in jest-enzyme README.md

setupTestFrameworkScriptFile was deprecated in favor of setupFilesAfterEnv.

Note: setupTestFrameworkScriptFile is deprecated in favor of setupFilesAfterEnv.

https://jestjs.io/docs/en/configuration#setupfilesafterenv-array

Thanks for bringing to my attention! This looks great. I'm curious, do we need to change anything for jest-environment-enzyme then? (https://github.com/FormidableLabs/enzyme-matchers/tree/master/packages/jest-environment-enzyme)

You’re right. I will do it in a separate PR if you don’t mind.


Walls cannot be lavalogged

no walls can be logged by fluid other than water

Fixed in 3.1.1/3.1.2.


💅 Style the Home Page

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

@all-contributors please add @atapas  for Code

@all-contributors please add @nirmalkc  for Code


Cards overwrite regardless of duration

Hey! Thanks for maintaining this amazing plugin for us!
Not entirely sure if you can fix this, but I notice the cards will follow the priority system to a fault in that it will overwrite the pre-existing cards that are already on someone. This is mostly important during initial opener, I suppose, so a minor issue, but I hope it can be addressed
Thanks!

You are a really Amazing Person !
thanks for your hard work


how to NON-HTTP Service Trace

Hello.
First thank you for your support.
I would like to trace for non-http service.
I've using scouter and trace non-http service with configured below
_trace_auto_service_enabled=false
_trace_auto_service_backstack_enabled=true
hook_service_patterns=comn.batch.framework.launcher.WebAppJobLauncher.runJob
hook_method_access_public_enabled=true
hook_method_access_private_enabled=true
hook_method_access_protected_enabled=true
hook_method_access_none_enabled=false
hook_method_ignore_prefixes=get,set
I tired to trace non-http service with using pinpoint which options like that :
profiler.entrypoint=comn.batch.framework.launcher.WebAppJobLauncher.runJob
runJob is Method.
But, pinpoint has couldn't trace anythings.
how can I trace non-http service ?

Sorry. I sovled it.
it's work after add config in release/pinpoint-env.config with profiler.entrypoint=comn.batch.framework.launcher.WebAppJobLauncher.runJob
thank you for read.

Great :)
Pinpoint needs a certain entry-point, if Pinpoint-Agent isn't attached ahead.