[BC Breaks] Stop providing prebuilt bundle for browsers. Drop bower support.

We have no reason to keep supporting prebuilt web bundle and bower.
refs #29
BREAKING CHANGE:
We stopped providing prebuilt bundle for browsers. Please build your own using browserify, webpack and so on.
We also dropped bower support. Please use npm instead.

Coverage remained the same at 97.716% when pulling c5a5f01570c6ea13a64b7734d380fddacafca1e3 on drop-support into 8a8f0e805833cd637e3325345818adadd76e2d3f on master.

Coverage remained the same at 97.716% when pulling 3eb82bec3efc551397faf756111c1402e96ccd8b on drop-support into 8a8f0e805833cd637e3325345818adadd76e2d3f on master.

Coverage remained the same at 97.716% when pulling 2771fe58e4077fd109614791278af6290dd54294 on drop-support into 8a8f0e805833cd637e3325345818adadd76e2d3f on master.

Coverage remained the same at 97.716% when pulling 1eca7fd92c2c8fbf39f9c2c091d21a06e3f8aa64 on drop-support into 8a8f0e805833cd637e3325345818adadd76e2d3f on master.

Coverage remained the same at 97.716% when pulling 1eca7fd92c2c8fbf39f9c2c091d21a06e3f8aa64 on drop-support into 8a8f0e805833cd637e3325345818adadd76e2d3f on master.

Coverage remained the same at 97.716% when pulling d6ce21161ffc016c697f110311a42fd383c7a421 on drop-support into 8a8f0e805833cd637e3325345818adadd76e2d3f on master.


Impr/file detection

Searches directory recursively

This resolves an issue where files are not automatically detected if all your files are inside folders

Don't initialize git

Avoid automatically initializing git in repository which might lead to git being initialized when goignore binary file is just needed to generate gitignore files. For example, I might have goignore binary file in my utilities folder to generate gitignore files to copy and paste inside my projects. Don't want to initialize git in that folder just to generate a gitignore file.

Run without installing package.

Update README.md file with instructions allowing users to use the goignore in other non-golang projects

You are welcome


/problems endpoint skip does not work independently

This is due to the nature of the LeetCode's GraphQL API. It allows the skip to be only the multiple of the limit. Because of the api shows it on the main page and a pagination issue will occur on the main page

A temporary fix is that, We set the limit to be 1 if only the skip parameter is provided

A permanent fix would be just like this guy does here https://github.com/bunnyxt/lcid
It triggers a workflow using GitHub actions daily to get all data for all the problems and store all the data into a json file. And the /problems route could then fetch from this file as the user wish to, from any skip to any limit


Specifying (async) generator behavior

Related: https://github.com/nodejs/node/issues/42237, https://github.com/open-telemetry/opentelemetry-js/issues/2951
Should a generator or an async generator capture the async context when being created? Given:
const context = new AsyncContext();

async function* gen(context) {
  await Promise.resolve();
  // ???
  console.log(context.get());
  yield;
  await Promise.resolve();
  // ???
  console.log(context.get());
}

let g;
context.run('init', () => {
  g = gen(context);
});
await context.run('first iter', async () => {
  await g.next();
});
await context.run('second iter', async () => {
  await g.next();
});

The output options could be:

'first iter'/'second iter', which is the context when the generator's next() is invoked.
'init'/'init', which is the context when the generator is being constructed.

My initial instinct is option 1, first iter / second iter... specifically, the context whenever next() is invoked but I do see cases where option 2 would be useful also. Unfortunately, with the current AsyncLocalStorage API option 2 is very cumbersome to implement.
const { AsyncLocalStorage, AsyncResource } = require('async_hooks');

const als = new AsyncLocalStorage();

function* gen(resource) {
  yield resource.runInAsyncScope(() => als.getStore());
  yield resource.runInAsyncScope(() => als.getStore());
}

let g = als.run(123, () => gen(new AsyncResource("abc")));

console.log(g.next());

I'd suggest that in the default case, it should be option 1 but we should have something defined that allows for option 2.

I think Option 1 is the correct choice, because it matches the sync calls-stack behavior that AsyncContext is based on.

I'd suggest that in the default case, it should be option 1 but we should have something defined that allows for option 2.

It's not too hard to define a wrapper that does this without you having to change your generator function:
const it = wrapGen(gen());
it.next();
it.next();

function wrapGen(it) {
  const getNext = AsyncContext.wrap(() => it.next());
  if (it[Symbol.iterator]) {
    return wrapSync(it, getNext);
  }
  return wrapAsync(it, getNext);
}

function* wrapSync(getNext) {
  while (true) {
    const next = getNext();
    if (next.done) return next;
    yield next;
  }
}
async function* wrapAsync(getNext) {
  while (true) {
    const next = await getNext();
    if (next.done) return next;
    yield next;
  }
}

Actually, we'll need a bit of spec test to handle parallel calls to an async generator:
const ctx = new AsyncContext();

async function* foo() {
  // Make it so the gen doesn't complete sync
  await sleep(100);

  console.log(ctx.get());

  yield;

  console.log(ctx.get());
}

const it = ctx.run(1, () => foo());

await ctx.run(2, () => it.next());
await ctx.run(3, () => it.next());

Which combination of 1, 2, and 3 should this log?:

1, 1
1, 2
1, 3
2, 2
2, 3
3, 3 Obviously not this one, right?

Well, currently it logs 2, 3. How about this setup:
const it = ctx.run(1, () => foo());

await Promise.all([
  ctx.run(2, () => it.next()),
  ctx.run(3, () => it.next()),
]);

This one logs 2, 2!
The queuing behavior of async generators is a little weird. It maintains its own queue of promises, but doesn't use HostCreateJobCallback. And, if there are multiple parallel calls, it will continue processing the queue.

To clarify, this would apply to both generators and async generators, or only the latter?

Both

While working on the spec text for this, I noticed there are various spec-created iterators which use the generators machinery under the hood through CreateIteratorFromClosure, and would therefore propagate the async snapshot from the time the iterator is created.
Now, this can only be observed if there's some user code running inside the generator, but in the spec as it is right now, that is already the case for array iterators (and I think also regex iterators) if some property has a getter:
const asyncVar = new AsyncContext.Variable();

const array = [23, 34, 45];
Object.defineProperty(array, 1, {
  get() {
    return asyncVar.get();
  }
});

const iter = asyncVar.run("foo", () => array.values());

iter[0] = 12;
asyncVar.run("bar", () => {
    console.log([...iter]);  // [12, "foo", 45, 56]
});

But with the iterator helper proposal, any iterator helper callbacks would run in the generator context:
const asyncVar = new AsyncContext.Variable();

const array = [23, 34, 45];
const iter1 = array.values();
const iter2 = asyncVar.run("foo", () => {
    return iter1.map(v => asyncVar.get()).take(1);
});
asyncVar.run("bar", () => {
    console.log([...iter2]);  // ["foo"]
});

Is this expected? Or should we somehow exclude CreateIteratorFromClosure from this behavior?

Would it be difficult to exclude? I don't think the behavior is problematic, but I would prefer consistency for the language's iterators.

I would prefer consistency for the language's iterators

Wouldn't the alternative be to make language iterators that don't use internal generators to manually carry the async context?

Is it reasonable to reopen this issue in light of discussion on #75?
To make this concrete, consider the following snippet:
const v = new AsyncContext.Variable();
async function* foo() {
  for (let i = 0;; i++) {
    await 0;
    console.log(v.get());
    yield;
  }
}
const it = v.run(1, () => foo());
v.run(2, () => it.next());
v.run(3, async () => {
  await it.next();
  it.next();
});

I posit two reasonable options for what this should log (independent of removing any combinations of async/awaits:

1, 1, 1 - this is the current state of the proposal, treating yield the same as await
2, 3, 3 - this has the context within the generator inherit from the caller of next(), and is closer to (see note below) the current behavior of AsyncLocalStorage on Node

I understand from discussion on the recent issue (#75) that @littledan and @bakkot were in favor of option 1, and indeed I initially preferred it.  On the other hand, @jridgewell and @qard have argued for option 2.  As I see it, some benefits of option 1 include

may be easier to reason about
more appropriate when passing a generator to an API (cf. https://matrixlogs.bakkot.com/TC39_Delegates/2023-03-23#L173)

Benefits of option 2 include

more flexible - you can make option 1 from option 2 with a userland wrapper, but not the other way around
easier to transpile/polyfill - no need for any special handling for generators
more consistent with existing AsyncLocalStorage behavior in Node
more consistent with existing built-in iterators (e.g. the array element getter example above)

I wasn't in the earlier discussions last year, but from what I can glean from the various meeting notes and matrix archives, it looks like "Committee prefers init time capture and restore" (from Jun 27 meeting) was in actuality a pretty weak preference?  Given the trade-offs above (and feel free to expand on any that I missed), I suspect it might make sense to pivot and go with option 2 here?  Or are there considerations I missed?
Note about AsyncLocalStorage: currently the code sample as written actually logs 2, 2, 3 on Node because of task queuing subtleties.  I'll argue that if option 2 is picked, we should fix this.

I hadn't seen @bakkot's example before, but I think it's flawed. Turning this into a "library takes a promise" example, we can see a similar context escape if the library ever needs to call back into user code:
async function libraryTakingAPromise(promise) {
  // Only the promise passed to us was wrapped, not the actual call to libraryTakingAPromise.
  // So, by chaining on the promise (which escaped its context), we've lost that context.
  await promise;

  // No way for user code to recover the context id, because this runs outside the `ctx.run()`.
  rerender();
}

libraryTakingAPromise(context.run('id', async () => {
  await 0;
  // Any chains on the async-fn's return promise will not propagate context id.
}));

The appropriate way to write that would work for both promises and generators:
context.run('id', () => {
  libraryTakingAPromise((async () => {
    await 0;
    // The library is capable of chaining on this promise just fine.
  })());
});

context.run('id', () => {
  libraryTakingAnIterator(context.run('id', function* () {
    // The library is capable of capturing context just fine.
    // If it were sync calling `.next()` nothing would be needed.
    // And if it was used `await`, then context would propagate already.
    yield 0;
  }));
});

For more context, I tried to do a [GitHub search](https://github.com/search?q=AsyncLocalStorage+getStore(+"function*"&type=code) for code that's using ASL with generators today.  As expected, I'm seeing a lot of middleware, which further suggests that there may be value in propagating context into next() calls, treating each iteration as a separate function invocation by default.  We should also clearly document how to make a wrapper (or provide one out of the box).

Turning this into a "library takes a promise" example, we can see a similar context escape if the library ever needs to call back into user code:

I'm confused by this transformation. I don't know what rerender is, but I don't see any reason to expect it to be called in the 'id' context - it has nothing to do with it. But for the generator example, the user is explicitly calling a (generator) function in the 'id' context, and so would expect its entire execution to be in that context.

I wouldn't expect the context to be bound through the flow of a generator - the same way I would not expect a variable in a try...finally (a sync context) to do so

I don't know what that means. What does try/finally have to do with anything?

I don't know what that means. What does try/finally have to do with anything?

With try...finally, I mean a pattern for maintaining a sync context through a normal variable
export let currentContext;

export function run(v, callback) {
  const parentContext = currentContext;
  try {
    currentContext = v;
    return callback();
  } finally {
    currentContext = parentContext;
  }
}

With this simplified implementation of sync context, a consumer can read the currentContext from anywhere, just like with the purpose of AsyncContext.run().

I don't know what rerender is, but I don't see any reason to expect it to be called in the 'id' context - it has nothing to do with it.

Sorry, my example wasn't great. But the point is that the library could reenter into user code, and due to the nesting the run call inside the library fn call, the promise escapes the context. A more concrete example of this is with a tracing function and an event handler:
// This is Library code.
function render(ui: Promise<JSX>) {
  // We wait for the ui to be ready.
  const vdom = await ui;

  // just imagine this does the normal calls to to a rerender a component tree…
}

// - - -

// This is User code
// Imagine we want to capture RAIL metric from click to rerender.
const rail = new AsyncContext.Variable();

function Button() {
  // This measure would probably wouldn't be in Button, but this is just an example.
  const span = rail.get();
  if (span) {
    // If we're rerendering, then we want to end the span now that we've painted
    span.end();
  }

  const onclick = () => {
    // We start the span during the user interation. But notice that `render` isn't wrapped,
    // only the UI generation. That means the `await` in `render` will not inherit our
    // context's span, and the rerender won't have a `span` variable to end.
    rerender(rail.run(new Span(), async () => {
      // Some work to generate a new UI… maybe a network request or whatever.
      return <Button />;
    }));
  };
  return <button onClick={onclick}>click me</button>;
}

My point is that it's unlikely anyone intends to wrap only an async/generator function when they pass it to a library function. Instead, you'd wrap the entire call to the library function. In both async and generator functions, it'd be easy to propagate the context with that.

But for the generator example, the user is explicitly calling a (generator) function in the 'id' context, and so would expect its entire execution to be in that context.

I don't see this as different. If I call an async function in the 'id' context, then let the promise escape that context before continuing the computation (by awaiting/.then() chaining), then I've intentionally escaped the context.
function* gen() {
  console.log('before yield');
  yield;
  console.log('after yield');
}
const it = ctx.run('id', () => {
  const it = gen();
  it.next();
  return it;
});
ctx.run('other', () => {
  it.next();
});

async function asyncFn() {
  console.log('before yeild');
}
const p = ctx.run('id', asyncFn);
ctx.run('other', async () => {
  p.then(() => {
    // "resuming" the generator in a different context.
    console.log('after yield');
  });
});

If you want a generator to keep the same context, then consume the generator within that context. Eg, call it.next(); it.next(); it.next()… for a sync generator. Or await it.next(); await it.next()… in an async generator. Either way, both will propagate the intended context already.

With try...finally, I mean a pattern for maintaining a sync context via a variable

I'm not sure how this relates to the generator discussion?

I don't see this as different.

I think this is our core disagreement. To me the behavior you've written for async functions is totally expected: it has returned; its execution has finished; there is no reason to expect future work performed using the return value of that function to be related to the context that the function was called in. By contrast, the execution of the generator function is not yet finished; it has not yet returned. So it would be extremely surprising for the body of the function to suddenly be in a different context, since the whole point of this feature is to save and restore the context during the lifetime of a computation which may not complete in a single step without the consumer of that computation needing to manually manage the context.

Also, https://github.com/nodejs/node/issues/42237#issuecomment-1957227002 suggests Node at least still regards their current behavior (option 2 but IIUC with the Zalgo issue in https://github.com/tc39/proposal-async-context/issues/18#issuecomment-1463304093) as a bug, and that option 1 is the "intended" behavior.
So there's a couple data points to consider.

I suggest that when thinking about the correct behavior, we should consider async functions and sync generators as the two fundamental cases.  The behavior of async generators should fall out from merging the above two cases (though it's of course worth validating that the derived behavior is still reasonable).  Async functions is clear, so that leaves the question of what makes the most sense for sync generators.
If I write
function* gen() {
  a();
  yield;
  b();
}
const it = v.run(1, gen);
v.run(2, () => it.next());
v.run(3, () => it.next());

which contexts do I expect within the body of the generator, when a() and b() execute?  Unfortunately, I think the answer here depends heavily on the situation.  I believe @bakkot is correct that in the majority of cases, the intended behavior is for everything to run in context 1.  However, in https://github.com/tc39/proposal-async-context/issues/75#issuecomment-2008119267 @qard brought up a use case where generators are used to model a work queue of independent tasks - in this situation, next() is effectively treated like an ordinary function invocation, which one would expect to snapshot the current context, rather than restore the generator's initial context.  The example was also interested in tracing (similar to https://github.com/nodejs/node/issues/42237#issuecomment-1957227002), but in that case it was important to retain the trace surrounding the next(), rather than the initial one.
In considering this, it seems like the question becomes whether it's more important to make option 2 possible at all, or whether it's more important to make option 1 an easy and fast default.  If we go with option 2, then option 1 is quite possible with a wrapper around the generator (as demonstrated in the nodejs issue).  If we go with option 1, then option 2 becomes completely impossible.  So, what are the relevant trade-offs to consider here?
To @dead-claudia's point about performance - it's a little surprising to me (though still understandable) that the performance would be noticeably worse for async generators when it seems to naturally be doing less work.  Snapshotting and restoring context for async functions is additional work on top of what would happen without AsyncContext, so being able to skip that extra work for yield seems at first glance like it should be a win.  Indeed, when transpiling code to support an AsyncContext polyfill, it's certainly less work to not have to do the extra snapshots/restores.  So I would expect at least sync generators to be no worse under option 2, and maybe even better.  But I can also understand how the combination could cause extra complexity due to the fact that it's now a different snapshot being restored by the awaits at different points in the body.
Finally, a brief further word about async generators.  When I searched for generators declared in the same file as AsyncLocalStorage usages (https://github.com/tc39/proposal-async-context/issues/18#issuecomment-2010563961), I found that nearly all cases were actually async generators.  So for whatever it's worth, these seem more common than sync generators.  Most of the results seemed to be middleware, so I could imagine there's a possibility that they have more the characteristics of the "work queue" that @qard mentioned.  But without understanding the relevant frameworks, I can't say conclusively.  What unique considerations fall out from making a generator async?  In terms of common usage, a for-await-of loop will work the same either way, just like a for-of loop is going to behave identically under both options (assuming the generator is called in the loop initializer).  So the most likely scenario where this decision will make a difference is when somebody is interacting with an iterator manually.  This is already somewhat of a "power user" case, and given that, I tend to prefer to err on the side of flexibility, rather than worry about making sure the default is correct.  This is why I have a slight preference for option 2, assuming the performance cost doesn't make it a non-starter.

@shicks To be clear, the perf impact, if it does exist, would most impact async functions and sync generators. The request queueing overhead would make it less significant for async generators.
Also, this overhead can be easily avoided if you duplicate the code (say, via three different template instantiations), and JITs can of course generate specialized code for each one - this trades some minor CPU cost for a significant growth in binary size. Size-optimized runtimes running on lower-spec devices like low-end phones and (especially) microcontrollers won't be able to specialize away the overhead like this, though, especially if JIT compiling isn't an option.

I looked into https://github.com/nodejs/node/issues/42237, and the linked nice-grpc fix linked in https://github.com/open-telemetry/opentelemetry-js/issues/2951#issuecomment-1214587378. It seems people really are going to return the generator's iterator without consuming it themselves. 🫤
It also has code using yield * call.foo(…), which itself could be a middleware generator, which can't be consumed within a context without writing a bunch of manual iteration code. You could certainly wrap call.next(…), but I would probably write that code and miss the bug.

I expect the performance of either choice to be trivial, I don't think it's something that we should consider.

If we were to do init-time, how could we support call-time? Given that the generator may already expect an argument when calling it.next(arg), we probably would need to pass it as an arg to the generator:
function* gen(snapshot) {
  snapshot.run(…);
}

let snapshot = AsyncContext.Snapshot();
const it = gen({
  run(...args) {
    return snapshot.run(...args);
  }
});

it.next();
ctx.run(1, () => {
  snapshot = AsyncContext.Snapshot();
});
it.next();

It's not great to support this way. If we really want to support it, maybe we could expose another way to start the generator (maybe genFn.noContext(…);) that wouldn't set the internal [[GeneratorAsyncContextSnapshot]] field when starting the generator.

It's not great to support this way. If we really want to support it, maybe we could expose another way to start the generator (maybe genFn.noContext(…);) that wouldn't set the internal [[GeneratorAsyncContextSnapshot]] field when starting the generator.

I think it's important to consider this use case.  One big problem with a noContext method is that there's no way for it to work (intuitively) for instance methods: instance.generatorMethod.noContext(...) may want to access instance as this, but such behavior would be very surprising (and a nightmare to polyfill, at that).
I feel like it also moves the decision to the wrong place - generally I would expect the author of the function know what context it expects, rather than the caller - so I'd rather see the decision cemented at definition time, rather than call time (i.e. by wrapping the function somehow - this could be done with a decorator, maybe - though again, I don't see a great way to get the right default).
Probably if the default were to snapshot at init and I wanted a generator that captured the snapshot, I'd end up writing something along the lines of
function wrapNoContext(fn) {
  return function() {
    let snapshot;
    const fauxSnapshot = {run: (...args) => snapshot.run(...args)};
    const gen = fn.apply(this, [fauxSnapshot].concat(arguments));
    return {
      next: arg => (snapshot = new AsyncContext.Snapshot(), gen.next(arg)),
      return: arg => (snapshot = new AsyncContext.Snapshot(), gen.return(arg)),
      throw: arg => (snapshot = new AsyncContext.Snapshot(), gen.throw(arg)),
      [Symbol.iterator]() { return this; },
    };
  };
}

and the usage would look something like this:
const gen = wrapNoContext(function*(snapshot, arg1, ...) {
  ...
  yield;
  ... snapshot.run(var1.get()) ...
})

gen(arg1, ...)

It's basically the same ideas as yours, but you can at least wrap up the usage sites to "just work".  Unfortunately there's no way to have it just work inside the generator definition, and you still end up needing the extra parameter, and to bend over backwards to actually extract anything out of the context.
@TerribleIdeaBot said:

I don't suppose there's any way to write the spec such that some sort of "indirection context" would be natively supported, without a huge performance penalty even when it's not used?

Note this Node.js bug report where people seem to be asking for the restore-after-yield generator behavior: https://github.com/nodejs/node/issues/52317

Do we have a conclusion for this thread? @jridgewell seems to have written for our April 2024 update slides that we're going to save/restore across yield in JS code, but not in built-in generators, which is a conclusion that I support. We have this all specified, e.g., in #61. What is left to resolve?

If we do #77 the decision to exclude generators may no longer be relevant as the calling context could be used to get the context where gen.next() is called. May be worth waiting on a decision for #77 before we make a decision here.

Our agreement from last group meeting was that landing #77 was an acceptable compromise so that generators can use init-time by default.

Form library developer perspective, I will return string or iterator of tokens for a AI tools. And I use async local storage to track which llm is being called and will do logging. It’s very to trick to handle iterator and I had to wrap every response (hopefully there is function decorator in standard)
So I will go with option 2. but another side this behavior looks like v8 inherited, there could have more breaking changes I think


Add uniq() builtin

Not sure if usable or not, but I implemented this recently in my own project (source + docs)
and inline below
func UniqSlice[T cmp.Ordered](in []T) []T {
	slices.Sort(in)

	return slices.Compact(in)
}

// Uniq takes a list of strings or interface{}, sorts them
// and remove duplicated values
var Uniq = expr.Function(
	"uniq",
	func(args ...any) (any, error) {
		switch elements := args[0].(type) {
		case []any:
			var result []string

			for _, element := range elements {
				result = append(result, fmt.Sprintf("%s", element))
			}

			return UniqSlice(result), nil

		case []string:
			return UniqSlice(elements), nil

		default:
			return nil, fmt.Errorf("invalid input, must be an array of [string] or [interface], got %T", args[0])
		}
	},
	new(func([]any) []string),    // []any -> []string (when using map() that always return []any)
	new(func([]string) []string), // []string -> []string
)

I couldn't figure out how to make the input accept cmp.Ordered instead of specific types :)

This is one of the limitations of our virtual machine. As we use stack of []any, information on type is "lost" (can be retrieved via reflection).
For uniq() builtin to work not only with strings, we need to implement a more general approach, probably implementing in in bytecode or in VM.
Implementing uniq() is much ore tricky 👀

Could u pls be more concrete and provide at least signature?)

Fixed in #705


Fix typos in AutoConstructedTypes.cs summary tags

Description

Summary tags for several classes in the AutoConstructedTypes.cs file contained the sentence, "Collection used to indicate if the property was initialized was created by the SDK." The phrase "was initialized was created" is likely a typo. I removed the redundant clause "was created," so that the documentation now reads, "Collection used to indicate if the property was initialized by the SDK."
Motivation and Context

Fixes a typo in the documentation.
Testing

N/A. Small documentation typo fix.
Screenshots (if appropriate)
N/A
Types of changes

[ X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist

[ X] My code follows the code style of this project
[ X] My change requires a change to the documentation
[ X] I have updated the documentation accordingly
[ X] I have read the README document
[ ] I have added tests to cover my changes
[ X] All new and existing tests passed

License

[X ] I confirm that this pull request can be released under the Apache 2 license

Thank you for your PR. I have this staged internally for an upcoming manual release of the SDK (TBD, as opposed to the daily release with service updates).

Thanks again for the contribution! This has been included in version 3.7.1 of AWSSDK.Core.
https://www.nuget.org/packages/AWSSDK.Core/3.7.1


enable docker run microsoft/iis cmd

enable
docker run microsoft/iis cmd

I'm looking at Apache and Nginx as well and does seem to be commonplace to specify the "entrypoint" of a base image using the CMD directive making it easy to overwrite.
@MichaelSimons @glennc Thoughts on making this change?

As I've seen the nginx, httpd and mongo official Dockerfiles, I would also prefer to use CMD here.
@friism any suggestions how to share best practices on both Linux and Windows?

What is the version scheme for this image and the potential impact this change may have on existing users?

Versioning scheme for this repo is microsoft/iis:[os-flavor]-[version]
Seems like a low impact change, however the ASP.NET images depend on this image they will be affected as well.

Are there ever any arguments you would want to pass to ServiceMonitor?

ServiceMonitor is generic enough to work with any service (that listens to Ctrl+C), but there's really no reason monitor any other service in the IIS/ASP.NET images.

Actually mongo uses ENTRYPOINT instead of CMD. https://github.com/docker-library/mongo/blob/b37a4891feffeafb77febd2833d96b59cf28d6a8/3.0/Dockerfile#L51 There doesn't seem to be a consensus everywhere. From article https://www.ctl.io/developers/blog/post/dockerfile-entrypoint-vs-cmd/. The dockerfile was authored with the idea that user shouldn't need to modify the entry point. @tg123, could you provide some more detailed description of your motivation for the change?

 This currently works

Hi @shufan
I checked the Dockerfile you referred, it uses CMD not  ENTRYPOINT.
COPY docker-entrypoint.sh /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

EXPOSE 27017

CMD ["mongod"]

As I know most docker images use entrypoint to do some init works that must be called when container starts.
The "ServiceMonitor" here works like a daemon on Linux, as a result, I prefer to use it in CMD.
And any detailed reason why user should not change the entrypoint?

Technically docker-mongo uses ENTRYPOINT and CMD is only there to provide the input argument.
It's true that ServiceMonitor serves like a daemon currently therefore CMD would make sense. However I know that there's some work going on for w3svc to be called directly. In that case ENTRYPOINT makes sense over CMD. So this is another reason that ENTRYPOINT was chosen.
@shirhatti should have more context on this


How to use with multiple accounts?

I would like to use this with my personal and work accounts, how can I do that?

Currently this script only supports a single account, you could add your work account calendar to your personal account calendar, then sync your personal calendar. This will make the script go through both calendars

This is not going to be good enough workaround. In case of a Google Account managed by an organization you can share with your personal account only with permission to see busy / free events :disappointed:

we could add an option to provide the script with multiple credentials file (using the -c flag)
This will require users to download 2 files, one for each account, then the script will simply fetch n times for each credentials file provided, it will then just merge the resulting lists and from there it's business as usual.
TODO:

Change the get_events call to be called for each credential file provided
https://github.com/rosenpin/i3-agenda/blob/master/i3_agenda/__init__.py#L33

Add support for multiple credentials file command line argument
https://github.com/rosenpin/i3-agenda/blob/master/i3_agenda/config.py#L14

It's now possible since @notdodo commit
Previously if you did it, both blocks would have pulled from the same cache.
Now you can set a different config dir for each block and let them run simultaneously
You would need something like this
Step 1
You will have to create both directories first
mkdir -p ~/.config/i3agenda/work && mkdir -p ~/.config/i3agenda/personal
Step 2
You will need to download both google credential files for both accounts and place them in the appropriate directories.
More information here
Step 3
Configure your bar
Polybar example
modules-center = agenda-personal agenda-work
....
[module/agenda-personal]
type = custom/script
; Show the next event and forget cache automatically every 60 minutes. Save cache in personal config dir
exec = i3-agenda -c ~/.config/i3agenda/personal/google_credentials.json -ttl 60 --conf ~/.config/i3agenda/personal
interval = 60
....

[module/agenda-work]
type = custom/script
; Show the next event and forget cache automatically every 60 minutes. Save cache in work config dir
exec = i3-agenda -c ~/.config/i3agenda/work/google_credentials.json -ttl 60 --conf ~/.config/i3agenda/work
interval = 60


Remove albumRipper.java

albumRipper.java is reduntent as everything it does abstractHTMLRipper also does. The rippers that use albumRipper.java should be rewritten to use abstractHTMLRipper.
The rippers that use albumRipper are :
src/main/java/com/rarchives/ripme/ripper/rippers/NewsfilterRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/TwitterRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/TumblrRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/ImgurRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/MotherlessRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/PhotobucketRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/VineRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/TeenplanetRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/RedditRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/PornhubRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/SmuttyRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/VkRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/NfsfwRipper.java

Looking at picking up TumblrRipper to rewrite. Is anybody already working on it?

@amolrbhagwat

Looking at picking up TumblrRipper to rewrite. Is anybody already working on it?

As far as I know no one else is working on it

@cyian-1756 Wouldn't a rewrite of RedditRipper use AbstractJSONRipper instead (or in addition)? I'm sorry if this is dumb question as I'm not fully acquainted with the codebase yet.

@aaether32323

Wouldn't a rewrite of RedditRipper use AbstractJSONRipper instead

Yup

Hello there,
I'm currently working on an assignment where I have to pick an open-source project and contribute to it. From the description, I really like your app (well, more the idea of it since I rarely download much stuff, though that could change :wink: ) and this issue seems to fit the assignments parameters (and most of my coding experience has been with Java).
I've checked the source code and the "API guide" (How to create a site ripper) on the wiki and am currently trying it out, but I have some questions before i fully start:

As there has not been much activity recently, what is the current status of this issue, is anyone assigned to it or working on it?
I have noticed that currently the test of PhotobucketRipper has been completely commented out due to issue #229 , so in effect, there is currently no test, so I was wondering if as part of solving the issue this test should also be reenabled and maybe fixed.

@PeterSzakacs

As there has not been much activity recently, what is the current status of this issue, is anyone assigned to it or working on it?

This issue is currently open to anyone who wants to work on it, there is no one currently assigned to it

I have noticed that currently the test of PhotobucketRipper has been completely commented out due to issue #229 , so in effect, there is currently no test, so I was wondering if as part of solving the issue this test should also be reenabled and maybe fixed.

The test failing is a separate issue

@cyian-1756
Ok, thanks for the answers, currently working on this issue. :)

Ok, guys, minor progress update.
Currently I have managed to rewrite the following rippers to inherit from AbstractHTMLRipper. All changes have been commited and pushed to my forks dedicated topic branch:

src/main/java/com/rarchives/ripme/ripper/rippers/NewsfilterRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/NfsfwRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/PhotobucketRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/PornhubRipper.java
src/main/java/com/rarchives/ripme/ripper/rippers/TeenplanetRipper.java

More details about the rewrite are in the relevant commit messages (1 commit per ripper). I've also done some basic testing and added or changed some tests as appropriate. Also, SmuttyRipper I have noticed already inherits AbstractHTMLRipper, so there was nothing to do there anyway.
I've also analyzed the source code for all other rippers, and next in line will probably be either VkRipper or TwitterRipper (the latter will use AbstractJSONRipper though).
I'm also writing to ask about further progress. As far as I can see, the individual rippers are independent from each other, and this task itself can be in theory subdivided into as many steps as there are rippers to refactor (plus probably an additional step for in-depth testing). Therefore, I am currently considering whether this issue won't be solved via multiple pull requests addressing only parts of this issue.
Normally, if this were possible to fix one ripper a day, I would not mind first finishing all rippers and only then submitting a PR, but I'm afraid these were the easy ones to refactor. While I have gotten quite confident with the codebase, all further rippers require some major analysis including probably manual API call tracing to figure out how they work currently (particularly Imgur ripper). As this would take a considerable ammount of time and taking into account the speed with which the project is developing (I'm already 12 commits behind upstream/master on my topic branch) as well as the fact that I'll be taking a break to focus on other assignments, I've been wondering if it would be a good idea to submit a pull request now (I can merge the latest commits into my branch locally, test the relevant rippers and push any needed changes back to my fork if needed).

I've been wondering if it would be a good idea to submit a pull request now

I imagine it would be.

Ok, guys, I've opened my pull request: #1040
Sorry it took so long, I was busy with assignments and my computer had some battery issues, so I couldn't use it until I fixed those. :)


Usage information text not correctly displayed via help script

When using help.sh the text is not displayed when selecting an usage information file. There is an empty window, but with a scroll bar on the right side which indicates that a long text seems to be inside the text field, but it is not visible at all (at least with dialog).
When using whiptail the text is displayed, but contains the ^M carriage-return character.
This problem occurred only on Linux Mint so far and maybe other Debian derivates.
Further investigations (e.g. versions, terminal emulator and settings) are still outstanding.

Fixed in version 1.22.1.


Build MVP pipeline for QLFS report

Pre-requisites

Need detailed working session on existing report

https://www.ons.gov.uk/employmentandlabourmarket/peopleinwork/employmentandemployeetypes/datasets/educationalstatusandlabourmarketstatusforpeopleagedfrom16to24seasonallyadjusteda06sa
https://www.ons.gov.uk/employmentandlabourmarket/peoplenotinwork/unemployment/datasets/unemploymentbyageanddurationseasonallyadjustedunem01sa
Drop overlapping months - starting at bottom
16-24 (or 18-24)

Added initial extract in https://github.com/open-innovations/yff-data/blob/main/data/qlfs/16_to_24_not_in_education.csv
Todo - long term unemployed

Largely working process - although need to discuss with YFF
Data in here - https://github.com/open-innovations/yff-data/tree/main/data/qlfs
Two files: 16-24 not in education and 18-24 long term unemployed

We've been looking at the A06 and UNEM01 spreadsheets, as discussed. These have been processed to the following files, tracked in our GitHub repo:

https://github.com/open-innovations/yff-data/blob/main/data/qlfs/16_to_24_not_in_education.csv
https://github.com/open-innovations/yff-data/blob/main/data/qlfs/18_to_24_long_term_unemployed.csv

These both sit in the same directory:
https://github.com/open-innovations/yff-data/tree/main/data/qlfs
Processing applied via this script: https://github.com/open-innovations/yff-data/blob/main/scripts/qlfs/transform.py
What's more important is the steps we're taking…

Load People sheet from Excel file
Filter by required columns (we're using the 4 character ONS codes)
Extract only the quarters that start Jan, Apr, Jul, Oct -- this is to be consistent with the NEET stats, as discussed.
Convert the string representation of quarter (e.g. Jan-Mar 2023) to a proper date time representing the start of the quarter.
Extract the most recent 3 years data, rename the columns based on this lookup that we maintain https://github.com/economic-analytics/edd/raw/main/data/csv/LMS_variable_lookup.csv and save as a CSV.

Processing for each file is the same.
The Q06 data is 16-24 age range. The UNEM01 data does not provide this combined range, so we have used the 18-24 set. It would be feasible to create an aggregate of 16-17 and 28-24 if this is desirable.

Feedback from YFF
Regarding the UNEM01 data, it might be helpful to look at 16-17 and 18-24.
Have added this in already in https://github.com/open-innovations/yff-data/commit/017cc3a2adee385ab07bdf96b6f71a83d5dc7107

Probably nothing more to do on this at the moment, with the exception of sensible column names! The current ones are horrible!

Have added rates for long-term unemployed and not-in-education from QLFS

YFF raised issue #52 following review. This is closed for the MVP


CORS errors

After editing the config file to match my server settings, I get this console error:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://<IP_ADDRESS>:9255/getPlaylist. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing). Status code: 200

Hello, Sorry for the inconvenience.
In new SonicClassifier_v0.9.2, cors_disable cofig has been added.
thank you


Fixed Hawkular console version

This builds locally with that command line:
mvn clean install -Pdev -DskipTests -Pdev

So I guess it is safe to merge anyway


[Epic][Test] Rewrite sample YAML tests with Golang

Description
Rewrite the sample YAML tests, which was originally written in Python, with Golang.
Reasons:

Those Python tests are very awkward when they need to interact with the Kubernetes cluster.
They are flaky tests.

Subtasks

[x] #2311
[x] #2431
[ ] #2451
[ ] #2452
[ ] #2453
[ ] #2454
[ ] #2455
[ ] Final step: Remove the tests folder in the repo root, sample yaml tests in CI, and Python environment setup in .buildkite/setup-env.sh

Sorry, wrong referencing


syslog-address not working in overlay networks

Rancher Version:
v1.1.0-dev5
Docker Version:
1.10.3
Setup Details: (single node rancher vs. HA rancher, internal DB vs. external DB)
HA rancher
Environment Type: (Cattle/Kubernetes/Swarm/Mesos)
Cattle
Steps to Reproduce:
When we add
log_opt:
    syslog-address: "udp:/karaf-logstash.logstash:5000"
    tag: "my-service/{{.Name}}"
log_driver: syslog

to our services in docker-compose we can't reach the logstash server by service name because of syslog-address not working in overlay networks.
I'm wondering if you have any workaround for this that aren't to complex?

Just to confirm, does your sys-log address have 1 or 2 slashes in it? I wasn't sure if you had copied and pasted what you used or if you had typed it up and just missed one.
When I get a chance, I'll try to reproduce this.

Oy, I need to check that when I am at the computer but I am pretty sure I used udp:// so hopefully it was a typo :P But I also read that syslog-address cant use overlay network because the docker daemon doesnt reach that network but maybe Rancher is doing some magic? :)

I used udp:// so that is not the problem

This seems to be the same issue:
https://forums.rancher.com/t/log-driver-linking-to-service-logstash-on-different-host-not-working/3216

This doesn't work because the log driver and config are part of the Docker daemon, not your container.  So you are "outside" of the overlay network, and also not using the internal DNS resolver that would know how to answer that.

Thank you for quick response!
How would you recommend to configure this?

Run 1 container of https://github.com/sendgridlabs/loggly-docker on each host (syslog + rsyslog).
Expose port 514/UDP.
In a service config, on "Security/Host" tab specify log driver = syslog.
Add "Log Option" syslog-address = udp://???????:514

How can I configure syslog service so that I can reference it in syslog-address?

Did you ever figure this out?  I'm been trying the same thing - no dice.
There seem to be a lot of "magic" container solutions with lots of documentation that don't work.   I don't think my use-case is unusual.   Simply funnel all containers STDIN / STDOUT logs to loggly.
I think the source of my issues is using RancherOS.  It doesn't have netcat, a package manager nor a configurable rsyslog.
I'm rambling, but can't find any workable solution and vested too many hours.

@ryanrca Yes, I did figure it out. Steps:
rsyslog service settings:

"Scheduling" tab: "Run one instance on each host".
"Security/Host" tab: check the "Full access to host" checkbox.
"Networking" tab: select "Host" from "Network" drop-down.
Make sure your rsyslog container listens on UDP: 514. Add this port on the port map in service configuration.

Your application settings

"Networking" tab: Network = "Managed".
"Security/Host" tab:

Log Driver = syslog
Log options: "tag"="app-name", "syslog-address" = "udp://<IP_ADDRESS>:514"

BTW, all of our hosts run Ubuntu Server, not Rancher OS. This shouldn't matter though, because the setup above is completely containerized.

With the release of Rancher 2.0, development on v1.6 is only limited to critical bug fixes and security patches.


ORCID Access Token and authenticated ORCID iD are not set correctly in migration

Describe the bug
When the migration routine is run on DSpace-CRIS version 2022.03.01, there are some problems with the ORCID. First, the ORCID is not set as authenticated, because the ORCID fields have moved from cris metadata schema to dspace metadata schema in version 2022.02.00.
To Reproduce
Steps to reproduce the behavior:

Migrate an old DSpace-CRIS 5 instance to DSpace-CRIS 7 with the migration scripts of version >= 2022.02.00
Look into the metadata fields of an eperson - there will be no field dspace.orcid.authenticated
Look into the database - the table orcid_token will be empty

Expected behavior
The field dspace.orcid.authenticated should be set for persons, which had the ORCID authenticated on the source DSpace-CRIS system.
Workarounds
dspace.orcid.authenticated can be set in the migration_configuration by mapping it from the old ORCID field in the metadata tab of the configuration excel. This will solve the problem for the non-authenticated iD. But the access token will still be lost and cannot easily be restored.
Another valid workaround: use the old migration procedure from DSpace-CRIS 2022.01.x and after the migration has been run successfully, upgrade the system to a newer version. This way Flyway will automatically correct the fields (but you will have to create a working configuration for the old system before you can do that).
Related work
None I am aware of.

More some workaround - and you should exactly know what to to when you pick it -  is to comment the deletion of the cris.orcid.accesstoken and eperson.orcid.accessToken metadatafields in the flyway migration and trigger the processing into the orcid_token table in the post migration skript (after the cris.orcid.accesstoken field has been filled in the migration). Because it's more some workaround and includes some changed to the flyway migration i will only link to the commit and not open some PR. https://github.com/uniba-ub/DSpace/commit/a70bc9bd8fb827f6afb4ccd10b963346c64dc2bc


Property:highestBiostratigraphicZone

Definition
The full name of the highest possible geological biostratigraphic zone of the stratigraphic horizon from which the cataloged item was collected.

Dimension

Existing property
dwc:highestBiostratigraphicZone

Existing class
Geological Context

Existing property identifier
http://rs.tdwg.org/dwc/terms/#highestBiostratigraphicZone

Format
controlled vocabulary

Required
No

Repeatable
No

Constraints

Examples
Blancan

Notes

vocabulary https://timescalefoundation.org/index.html


Spinner wrap

Would you mind adding a binding, say set_wrap(bool) and wrap() -> bool for Spinners please?

Thanks, I've made use of it and it works great.


Failing to get available flights

I'm successfully logging into southwest with the script and getting my future flights, but in the following code, available is returning with "None" as if the URL request in get_available_flights() didn't work.  This started abruptly last week after running successfully the several times the day before.  Anyone else having this issue?  Is this request also happening "logged in" to my southwest account, or is it a completely different request of the mobile API without the headers?
available = southwest.get_available_flights(
                        departure_date,
                        origin_airport,
                        destination_airport
                    )

I updated it a few days ago -- the request URL changed.


Replace FilePointer with universal pathlib

See https://github.com/astronomy-commons/hipscat/pull/336
Solution Description
Use fsspec's project, universal-pathlib as a path provider. In this way, everywhere we currently pass in a FilePointer we can use a Path instead, and when we need to create remote connections, use UPath.

Documentation issue will require https://github.com/astronomy-commons/hipscat-import/pull/387/, but I'd like to send for review for general comments.


Network Connectivity: Hidden Networks (App)

As a robot user, I would like to be able to supply a network id when connecting to wifi, so that I can connect to hidden networks.
Acceptance Criteria

User able to supply an ssid to connect to a hidden network
Question: do users have to be able to submit a security type? @mcous follow up with @sfoster1

Implementation Notes

Should we implement as a two-step flow?

Closing in favor of #2134


Initial proposal about how to write Proposals

Looks good to me, I like the script to help generate the templates.. I'd like to make sure that the barrier to entry isn't too high that it scares away new contributors to the project also. Is there additional methods (more scripts or templates, or an instructional video etc..) that a new person would benefit from to use this process?

@thebsdbox ideally this is the place where we agree on the process and from there we will have to figure out all the work that has to be done in order to move this RFD into a committed state. One will be as you suggested:

[ ] write READE.md for this repo explaining steps required to write a CFD copy-pasting in there what is needed in order to make the process super easy and clear for the community

Another will be for example:

[ ] Set up a code generated site called cfd.tinkerbell.org that lists the status of the CFD

This sounds good to me. 👍🏻

Thank you all for your feedback.
The proposal number 0001 is ready to go. It is in published state, it means that it can be modified via PR. The status will change to committed as soon as we will have a static site generated version of this REPO available to proposals.tinkerbell.org


Download all non-local Unsplash Images and Compress them for use in the Website

Right now, the images in Section Two (Why Invest in Africa) and Section Five (Industries) are direct links to Unsplash Images

This is very slow. It slows down the image loading of the site by a ton. Instead we need to download them onto our repo, compress them, and import those images for use.

The downloaded images must be placed in a logical place. (As in, in the assets folder in their respective folders)

Objectives:

[ ] Open each Unsplash link for each image, and download the image (if possible - medium quality, not higher or lower)
[ ] Open https://squoosh.app/ and paste the image in. Follow the settings in the image below exactly
[ ] Set the left side as the original image, and the right side as WebP. 50% quality, 6 Effort. Do not turn anything else on. Click the blue download button
[ ] Place all the new WebP images in the respective folders
[ ] Import all new images into the repo and replace all the Unsplash links with them
[ ] Test and ensure all the images work

Asignee - Sameer


typo

forgot the E for AzurServicePrincipal. It is now AzureServicePrincipal

@jrudley : Thanks for your contribution! The author, @georgewallace, has been notified to review your proposed change.

@jrudley Thanks for the contribution and appreciate your catching that typo!  #sign-off

@DCtheGeek: I'm sorry - only the author of this article, @georgewallace, can sign off on your changes. But we do have an exception process - if you are on the Microsoft content or product team for this product area, you can ask the PR review team to review and merge it by sending mail to the techdocprs alias.


Is it possible to scaffolding with other individual HiC data?

Thank you for developing such an excellent software! I am currently using Triobin to assemble offspring individuals
yak count -k31 -b37 -t16 -o pat.yak <(zcat $paternal) <(zcat $paternal)
yak count -k31 -b37 -t16 -o mat.yak <(zcat $maternal) <(zcat $maternal)
hifiasm -o $prefix -t 32 -1 pat.yak -2 mat.yak $child_hifi)

but I don't have Hi-C data from the same individual for scaffolding. Instead, I only have ~80x Hi-C data from another individual. Could you please let me know what potential issues might arise from doing this?
I tested the results of combining two haplotigs and processing them separately, following the official recommended workflow.
processing them separately

bwa mem -5SP -t 60 $genome $Hic_1 $Hic_2 | samblaster | samtools view - -@ 60 -S -h -b -F 3340 -o HiC.bam
filter_bam HiC.bam 0 --nm 3 --threads 60 | samtools view - -b -@ 60 -o HiC.filtered.bam
haphic pipeline $genome HiC.filtered.bam $nchr

I noticed that when following the recommended filter_bam step (MAPQ>=1), the remaining data in the combined processing approach seemed overly reduced, which might suggest insufficient heterozygosity? Or perhaps the use of Hi-C data from a different individual resulted in a large amount of sequence being filtered out (Hi-C BAM file size reduced from 156 GB to 17 GB after filtering). As a result, I abandoned using the combined haplotigs approach. The results from processing the two haplotigs separately seem to have some minor issues, but the problems don't appear to be very significant.
The main issues in the results of both hap1 and hap2 are concentrated on the relatively complex sex chromosomes of this species.
hap1 suspicious sex chromosomes (I moved the lower right suspicious sex chromosome to the upper left with the interaction)

hap2 suspicious sex chromosomes

However, the overall Hi-C map doesn’t seem to indicate significant problems.
hap1 whole genome

hap2 whole genome

Could I use Hi-C data from another individual for scaffolding? Do you have any suggestions for assembling such complex sex chromosome structures? Can I use the human HG002 parameters when assembling the two haplotigs?
I would appreciate your reply!

Hi @YPGG1234,
The trio binning mode of Hifiasm can typically yield high-quality haplotype-phased assemblies, with contigs from each parent already output into two separate GFA files. At this point, it may be feasible to perform scaffolding using Hi-C sequencing data from another individual, but this would require scaffolding each haplotype separately. This is because the haplotype composition differs between individuals and cannot correctly guide haplotype-phased assembly/scaffolding.
For the strategy of scaffolding each haplotype separately, I can think of three potential issues:
There may be chromosomal structural variations between different individuals, which could lead to incorrect ordering and orientation of contigs.
Similarly, within a single individual, there can also be significant structural differences between homologous chromosomes, especially between sex chromosomes.
If sex chromosomes have a structure similar to that of the human genome (with PAR and non-PAR regions), this can also lead to certain issues. Firstly, the PAR regions, due to their high similarity, may introduce errors during the upstream assembly and phasing processes. For instance, it's possible that the PAR regions of the two sex chromosomes might be incorrectly assembled into a single sequence (forming collapsed contigs). Moreover, due to low homology, the non-PAR regions of the two sex chromosomes might be incorrectly phased into the same haplotype by upstream assembly software (phasing error). Secondly, because the non-PAR regions exhibit significant differences, their actual sequencing depth is only half that of autosomes and PAR regions. This would be reflected in the heatmap as lower interaction signals for this region.
In summary, performing phased scaffolding without Hi-C data from the same individual presents challenges, especially when sex chromosomes are involved. Whether this problem can be fully resolved also depends on the characteristics of the species, and at present, I cannot provide a definitive answer.
Best,
Xiaofei

Thanks for your quick and detailed reply! I knew it would be better to use hic data from the same individual. I know from your reply that it is better to merge the haplotigs together for scaffolding. I was wondering if scaffolding the merged haplotigs with hic from different individuals would cause the filter bam to become very small, rather than responding to low species heterozygosity? If the merged haplotigs are scaffolded with hic data from the same individual, does the filter bam retain more data.

This should not be the main reason affecting the size of the BAM file. The main factor is still the heterozygosity level. If the heterozygosity level is too low, many Hi-C reads will have multiple alignments and will be filtered out by the MAPQ>=1 criterion. Additionally, if the genome sequence differences between different individuals are very large, such as exceeding 2-3%, the --nm parameter might filter out many alignments due to mismatches.

Thank you for your reply!


Error: Cannot find module './certificate'

I get this error since latest version, when installing on Azure Web Sites. Before it was working.
Application has thrown an uncaught exception and is terminated:
Error: Cannot find module './certificate'
    at Function.Module._resolveFilename (module.js:325:15)
    at Function.Module._load (module.js:276:25)
    at Module.require (module.js:353:17)
    at require (internal/module.js:12:17)
    at Object.<anonymous> (D:\home\site\wwwroot\bot\node_modules\sshpk\lib\fingerprint.js:10:19)
    at Module._compile (module.js:409:26)
    at Object.Module._extensions..js (module.js:416:10)
    at Module.load (module.js:343:32)
    at Function.Module._load (module.js:300:12)
    at Module.require (module.js:353:17)

@arekinath Thank you, I have found that it was like a continuos  deploy issue on the Azure Web Site machine, checking the console after several attempts, I get all files there

Now it works ok!


v0.5.0 release candidate branch

Summary

This PR is for the v0.5.0 release candidate.

Please check your committed PRs against the draft release notes and update the release notes to capture all notable changes that users should be aware of. Check the box next to your name below when you have verified that the release notes properly account for your contributions:

[x] @white238

[ ] @cyrush

[x] @kennyweiss

[x] @nselliott

[ ] @ltaylor16

[x] @agcapps

[x] @bmhan12

[x] @joshessman-llnl

[x] @rhornung67

In RELEASE-NOTES.md, there are lists of "Added," "Fixed," and "Changed" items.  Do newest items go on the top of these lists, or the bottom?  I have one item to add, and I'd like to put it (close to) the right place in the list.

I was going to merge develop into this branch then start the release process.
Any objections?

@ltaylor16 It looks like all the PRs we wanted to have in have been merged. So, unless someone objects, I think you can merge develop into this branch. Before starting the release @nselliott can you double-check the release notes to make sure they include the SCR updates. Also, @white238 @kennyweiss if there are any build system changes that users should know about based on a newer version of BLT, please make sure that they are captured in the release notes.

Thanks @nselliott !

@ltaylor16 when you change this PR to be merged into main for the release (next step), there will be a huge number of modifications since main hasn't changed since our last release. To make it easier to review a release candidate PR like that, I usually make a corresponding PR to merge the RC branch into develop and mention it in the PR to merge the RC branch into main. It will only contain the changes related to the release prep. Tell folks to review the changes in the PR to merge into develop, but approve the PR to merge the RC branch into main. There are other ways to do this as well, but this one seems the easiest to me.


Add logging

Adding logging will be an important method for tracking jockey usage.  We need to understand the frequency of usage as well as the kinds of commands that are run.  We can probably put logs in /var/log/juju/jockey, or something like that.
Logging throughout would be nice, but logging on CLI calls is essential.
It would make sense to add logging levels as a CLI option, once we know logging works.

Unlinked #47 since it's only CLI logging.


Build deploy actions

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have signed the Adobe Open Source CLA.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

@purplecabbage @Himavanth there was a bug reported by @kptdobe where building an action with an include and only a src is failing, e.g.
       include:
            - ["myfile"]

will throw
 TypeError [ERR_INVALID_ARG_TYPE]: The "path" argument must be of type string. Received type undefined
    Code: ERR_INVALID_ARG_TYPE

My commit above is a fix for this


Failed Context Types: Required context routeDepth was not specified in RouteHandler.

I have checked and I do not have React being instantiated more than once;
Using react-router v0.13.3
I don't think this would be very easy to reproduce. But I was wondering if you could give some suggestions on how to debug this particular error.
var { Route, Redirect, RouteHandler, Link } = Router;

var App =  React.createClass({
    render: function () {
        return (

                <RouteHandler/>

        );
    }
});

probably means your app is not being rendered with the Handler that v0.13 gives you. Please check the examples and make sure your'e rendering your app correctly: https://github.com/rackt/react-router/blob/0.13.x/examples/dynamic-segments/app.js#L67-L69


Device Info(?): POST(?) /setup/send_log_report

Found in this Chromecast support thread. Sorry, I have zero idea what should be POSTed to it or what it's actually for beyond what can be guessed from the name. Attempting a GET results in Error 405: Method Not Allowed, but an empty or invalid POST just results in 404 Not Found. Perhaps it's neither GET nor POST?
EDIT: Based on this page which I do not understand at all as a lay user, it seems to have something to do with sending crash logs to Google? And perhaps it's triggered by POSTing your UUID to that endpoint? From what I can tell, end users are not supposed to know their UUIDs, so I'm not sure how you'd find that info, assuming I'm even remotely in the ballpark of what this is all about. Quoting from that page in case it dies:
d.getAllResponseHeaders=function(){return this.ea&&this.Ty()?this.ea.getAllResponseHeaders():""};d.gH=function(){return r(this.Nh)?this.Nh:String(this.Nh)};d.pg=function(a){return a+" ["+this.vJ+" "+this.lp+" "+this.Sa()+"]"};var Bf=function(a,b,c){this.source=a;this.type=b;this.message=c};var Cf=x("mr.DongleUtils"),Df=function(a){return"https://crash.corp.google.com/samples?stbtiq="+a},Ef=function(a,b,c){var e=new tf;e.dO("text");e.Mu(3E5);Xc(e,"complete",function(a){a=a.target;var e,f;a.ie()?(e="ok",f=Df(b)):(e="error",f="Unable to retrieve "+a.ws()+", error = "+a.gH());c(e,f);a.Ya()});e.send("http://"+a+":8008/setup/send_log_report","POST",JSON.stringify({uuid:b}),{"Content-Type":"application/json"});return Df(b)},Ff=null,Gf=function(a){if(!r(a))return Promise.resolve();if(Ff&&Ff.ip==

Not sure about this yet. Need to check the app if it can shed some light.

Didn't find anything. Will reopen if someone gets more info.


Convert files in the src/display/-folder to use optional chaining where possible

By using optional chaining, see https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Optional_chaining, it's possible to reduce unnecessary code-repetition in many cases.
Note that these changes also reduce the size of the built pdf.js file, when SKIP_BABEL == true is set, and for the MOZCENTRAL build-target that result in a 0.1% filesize reduction from a simple and mostly mechanical code change.

/botio test

Thank you for doing this!


Add support for arrays, collections and compositeData

From https://github.com/elastic/logstash-contrib/issues/128 for array and collection

It's really cool to be able to have an input plugin for JMX !
However, currently, not all data (arrays, collections) gets properly handled:
examples:
[arrays]
"metric_path" => "java.lang:type=GarbageCollector,name=ParNew.MemoryPoolNames",
"metric_value_string" => "[Ljava.lang.String;@25647d72"
[collections]:
"metric_path" => "java.lang:type=GarbageCollector,name=ParNew.LastGcInfo.memoryUsageBeforeGc",
"metric_value_string" => "{#Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x4f722930=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x4fee370f, #Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x6091b85=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x3fedbbdb, #Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x49669be=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x2ef83b4c, #Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x1f12e637=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x11c834d1, #Java::JavaUtil::Collections::UnmodifiableRandomAccessList:0x73f5504=>#Java::JavaxManagementOpenmbean::CompositeDataSupport:0x4ee67d53}"

CompositeData support was requested in https://discuss.elastic.co/t/jmx-compositedatasupport-and-array/629

Is decorate event related to the conversion of array, collection, etc. into a standard format ?
I'm not familiar at all with jruby but to maybe help with that :
from what I understand in a blog post Struggling With Ruby, should the classes  javax.management.openmbean.* be loaded in ruby in order to properly map them with ruby object ?
And then it would be possible to recursively iterate through he object until it end up on simple object type which can be mapped ?
Or is there another approach which could be used ?

@mtarantino sorry but the decorate is just about applying the common logic for add_field and add_tag before the input send the event in the filter pipeline. Nothing done on this specific subject yet but you are right the specific Java object instances must be iterated using their java methods and converted into simple Array and Hash

I'm would like to try writing a small script to help a little. I just have technical issues when trying to iterate at the level of simple type (int, long, int[], long[], etc.). Instance_of doesn't work, class doesn't return a proper class name, etc.
What would be the "entry" point in order to iterate through all low level objects ?

Hi
I hope the following request fits in this enhancement, if not please advise.
I'm working on  a POC, to use this plugin to monitor the GC and Memory of our jvm's.
It fits our purpose to the point where composite data comes in.
What if we don't need all of the children of the attribute.
I did not find a way to filter on this level, using the json configuration.
And when going through the code of the jmx.rb file it always seems to do:
jmx_attribute_value.each do |jmx_attribute_value_composite|
(I'm not a developer, so I could definitely be wrong)
so as an enhancement request:
Could a filter on the level of the composite data of an attribute be added?
Using the same json configuration file to define those sub attributes we want to keep.


Fix workspace CSS properties syntax

[x] Bugfix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)

Proposed changes

Fixes the syntax of two CSS properties.
Checklist

[x] I have read the contribution guidelines
[ ] For non-bugfix PRs, I have discussed this change on the forum/slack team.
[ ] I have run grunt to verify the unit tests pass
[ ] I have added suitable unit tests to cover the new/changed functionality

Coverage increased (+0.02%) to 77.467% when pulling bd4fc2e5cc4337eded1913df390c1caed9721f35 on bonanitech:patch-1 into c9ad5bea93cf42a79165071069802fc2db513cd1 on node-red:dev.


Channels categorized as person

When I open a channel, it is automatically placed in "persons" after the opening


Modernize build2

Combo of #5 and #6.
@ghoshkaj and @chaupow - This brings in your working changes + node-cpp-skel best practices and tools. Per voice, in the future node-cpp-skel will provide cmake based build, but for now this is using gyp. I propose merging this to master, then working from this as a base.

Rebased this branch against congestion-cpp-hashmap. New plan is to merge this into congestion-cpp-hashmap and #6 into master. This will get master testing on travis without congestion related changes. And get the congestion work benefitting from the build modernization.

Codecov Report

:exclamation: No coverage uploaded for pull request base (congestion-cpp-hashmap@cc9134b). Click here to learn what that means.
The diff coverage is 0%.

@@                    Coverage Diff                    @@
##             congestion-cpp-hashmap       #7   +/-   ##
=========================================================
  Coverage                          ?   44.19%
=========================================================
  Files                             ?       11
  Lines                             ?      448
  Branches                          ?        5
=========================================================
  Hits                              ?      198
  Misses                            ?      250
  Partials                          ?        0

Impacted Files
Coverage Δ

src/hashmap.hpp
0% <ø> (ø)

src/types.hpp
0% <ø> (ø)

src/hashmap.cpp
0% <ø> (ø)

src/nodejs_bindings.cpp
46.37% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update cc9134b...3c6d188. Read the comment docs.

Closing. This work is now over at #8


Dissolve the for loop to str.join

Sting join is faster than using a for loop.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

LGTM


Source link is sometimes treated as an internal route rather than an external link in image detail views

Image detail view source links sometimes route users to a non-existent page within CC Search.
Sometimes, links in our database include the protocol (e.g. https://), and other times it does not. When the protocol is included, the user will be correctly routed, and when it is missing, they will be routed to the non-existent internal page.
This can be fixed by either:

Adding some logic in the frontend to account for inconsistent inclusion of protocols (stick 'https://' in front of the URL when required after fetching it from the API)
OR
Amending the database so that all source links include the 'https://' protocol prefix.

Example where the source link works correctly:
https://ccsearch.creativecommons.org/photos/92df8057-86e3-4d33-93fe-a51cb315d8ba
'Source' routes to https://www.flickr.com/photos/mikefats/387594564
Example where it's broken:
https://ccsearch.creativecommons.org/photos/ab41edc5-6075-4450-88a3-87bb99f629c0
'Source' routes to https://ccsearch.creativecommons.org/photos/www.flickr.com/photos/92082510@N04/15161614328

"creator_url" needs to be fixed for the same reason

The fix is in testing. I expect this will go out with the production 0.19.0 release on Friday.


feat:[CDS-96727]: review comments for jira kb for custom fields

Thanks for contributing to the Harness Developer Hub! Our code owners will review your submission.
Description

Please describe your changes: __________________________________
Jira/GitHub Issue numbers (if any): ______________________________
Preview links/images (Internal contributors only): __________________

PR lifecycle
We aim to merge PRs within one week or less, but delays happen sometimes.
If your PR is open longer than two weeks without any human activity, please tag a code owner in a comment.
PRs must meet these requirements to be merged:

[ ] Successful preview build.
[ ] Code owner review.
[ ] No merge conflicts.
[ ] Release notes/new features docs: Feature/version released to at least one prod environment.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66555528c7f8a98f4cb29eb7--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://66555560d828e985f4b44014--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://665555283247fc9ac2200639--harness-developer.netlify.app

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app. Current Draft URL is: https://6655663f3fd68db283e580d8--harness-developer.netlify.app


Implement properties arrays in the Inspector.

This implements property array support in the Inspector. It features multiple things:

move (via drag&drop or a right-click option), add, remove array elements,
clear or resize the whole array,
pagination.

Regarding the implementation, it adds a new usage flag PROPERTY_USAGE_ARRAY that can be used in two ways:

If the property it is assigned to is of type INT, the property is considered to be the array size. The class_name in PropertyInfo is then used to store, with a comma separator, the name of the array and the prefix of the array elements. For example, if your each property is numbered like "my_array/index_0/size", "my_array/index_1/size", etc.. then the prefix is "my_array/index_".
To simplify things I added ADD_ARRAY_COUNT and ADD_ARRAY_COUNT_WITH_USAGE_FLAGS that allows defining everything needed to define such a property. Note that, with the array set up using a count property, then the inspector will automatically try to detect and sort the properties belonging to each element. This is the simplest setup of the two solution, but it means that you won't be able to register more complex undo-redo steps than what Inspector can handle.

If the property is of type NIL, then you only need to give it a path for the array (if you want it to be somewhere in inspector hierarchy) and the array element prefix. The ADD_ARRAY macro was added for this case.
In this case you also need to give the inspector a function so that is can move the elements of the array (else you get a warning and things don't work), this allows more fine grained control on what the inspector will save in the undo_redo action. Registering a move function is done via: add_move_array_element_function(const StringName &p_class, Callable p_callable); // Function should have this signature: void (Object* undo_redo, Object *modified_object, String array_prefix, int element_index, int new_position). The move function should delete elements when new_position==-1 or create a new one when element_index==-1, otherwise, it should move the element from the index to the new position. This setup was widely used for the TileSet resources, as the properties exposed by the TileData objects in TileSetAtlasSource do depends on the TileSet layers. Using a dedicated function there allows keeping the TileData properties when removing a layer for example.

This new feature should allow getting rid of a lot of custom editors (like the custom ItemList items editor), or to simply expose things that were not exposed before (I'll open a tracker).
Note: as a first test I made the MultiMesh resource use this new feature. Even if the interface is optimized not to compute too much things, it means that I added a property for each transform/color in the resource. @reduz needs to have a look here to check the performance impact, but I might have to remove it.

The "+" button should likely have some text to it such as "Add Element".

The "+" button should likely have some text next to it such as "Add Element".

Like this ? I guess it helps:

Should there be a smaller button to remove the last element besides the Add Element button?

Should there be a smaller button to remove the last element besides the Add Element button? I'm not sure how often it would be used in practice.

I don't think it's needed, as you can already delete elements with the DEL key or using the right click menu.

Does this close https://github.com/godotengine/godot-proposals/issues/3059?


How to change the Reveal highlight color?

In the Readme file, the Calculator demo shows that some of the buttons have a blue reveal highlight color, but I can't work out how to go about setting this.
Thanks :)
Edit:
Worked it out for anyone else interested, it's not exposed directly; you need to change the following:
https://github.com/sourcechord/FluentWPF/blob/8ba57629fcefcb9652bb28e97ff5333bb9a81a41/FluentWPF/Styles/Button.xaml#L328-L333
to
             <ControlTemplate.Triggers>
                <Trigger Property="IsMouseOver" Value="True">
                    <Setter TargetName="root" Property="Background">
                        <Setter.Value>
                            <SolidColorBrush Color="{Binding ButtonBackground.Color}" Opacity="0.8" />
                        </Setter.Value>
                    </Setter>
                </Trigger>
            </ControlTemplate.Triggers>

Hi, @ImminentFate
Sorry for responding so late.
FluentWPF has two styles for Button control.(ButtonRevealStyle/ButtonAccentRevealStyle)
But, there is no property to customize the color when you move the mouse over the button.
I'd like to consider customizing hover color for Buttons.

It is supported from FluentWPF 0.8.0.
I'll close this issue.
If you have any problems, please reopen this issue.


Using SRID 4326 in vector-datasource

Is it possible to use SRID 4326 in vector-datasource? I understand I need to load the data into Postgis with SRID 4326, but what about querying the data? Should I change queries/*.pgsql files accordingly or there is more to accomplish this? My goal is to use an unprojected SRID in the database and project on the fly either at the application or at a client level. I don't want to use 900913 in the database, I'd like to store unprojected data in PostGis. And another question: what are the drawbacks with this approach?
Thanks!

It should be possible, but I haven't tried it so I'm not sure what the extent of the changes will be.
I expect that you shouldn't have to change the query files themselves, but the sql generation logic [1]. If the data is in 4326, you'll need to ensure that the bounds that are used in the query are in latlong instead of mercator coordinates. The other bit to keep in mind is that the formatters expect their data to be in a particular projection. The json/topojson formats expect latlong, and the opensciencemap/mvt formats expect the input to be in mercator.
[1] https://github.com/mapzen/TileStache/blob/9ddf505701c517efed56f7728e07e19bf17f9843/TileStache/Goodies/VecTiles/server.py#L500


Database.InsertAsync returns 0 on insert failure without an exception. (postgress)

Sometimes when inserts fail in postgres this function returns id of 0, but there is no exception thrown.
I would expect the Npgsql exception to be bubbled up.  It seems to happen if you are missing a non null column in your poco.
var id = await Database.InsertAsync(entity);
Might be related to InsertAsync not returning the right ID #361

What version are you using?

Sorry I missed that, Latest 4.0.2

Could you try 5.0.0-rc1? otherwise provide a small sample that reproduces the issue.


Avoid deep copies when saving post-hoc EMA checkpoints

This PR does two small things:

Avoid using deepcopy() when saving post-hoc EMA checkpoints.  This copy was being used to change the dtype of the checkpoint, but this can also be done by modifying the state_dict dictionary directly.
Add an option to never automatically write checkpoints.  In my application, I want to save a checkpoint every epoch, and it's easier to just call PostHocEMA.checkpoint() manually than to calculate the number of steps (accounting for batch size, DDP, and all that) that correspond to an epoch.

I know it's bad form to submit a PR that changes two different things, but I thought these would both be relatively uncontroversial.

@kalekundert looks good! left a comment!


Always initialize a variable to avoid -Wmaybe-uninitialized

This warning is sometimes given (e.g. when using optional) and can't be easily suppressed from the application code, so ensure the variable is always initialized to avoid it.
Closes #1037.

I've somehow completely forgotten to merge this, but it's done now.