Developer Landing Page

As an iterative step towards a more comprehensive developer portal, a developer landing page will provide a focused place where all the various developer-related resources can be found in a consolidated listing

Great job on this @bencgreenberg!
Content
To really set that page apart from the rest of the website and get that “developer-y” feeling more pronounce, we could focus on developer-friendly content first:

Remove the Getting started and Integrations sections as those are more about the product than the platform;
Split the Community section in two: “Join the community” with a link to the Discord so that folks know that they can reach out to other builders and ask questions, and “Built by the community” with links to open source projects built by the community. Build by the community could be among the first blocks as I find it a very strong signal as a developer, and a good thing to browse and get ideas from.
Change “Beta Features” to “Join the Beta program” to switch from passive voice to the active one. Can also mention that the beta program is a great way to suggest improvements to features/API before they reach GA.
That would require deeper changes (although totally feasible) but if we had tags on the blog we could link to our “Engineering” blog directly, i.e. we filter down by the “Engineering” tag.
We have a few engineering-focused talks by the Orbit team that could maybe find a home in this page as well? Yours as Rails Conf, mines at the Eleventies and Jamstack Berlin… Not 100% sure this is a good idea, just throwing it out there.

Design

Wondering whether we should remove the Orbit Model banner at the bottom as it’s standing out a bit too much currently
I riffed on an alternative to the content on each card:

Adding a few words of description with each item can let us be more explicit, or creative, or geeky etc…

Here’s the markup if you want to reproduce

API reference

View and test all endpoints in the browser

Key concepts

Learn about Orbit concepts and primitives

Common API calls

Handy code snippets for the most common requests

Swagger/Postman definitions

Import the API reference into your favourite tool

@phacks, thank you for such a thoughtful review. I don't think I've been greeted before by such deliberateness in consideration and in the formatting of it! ❤
Let me respond to your points:
Content

Remove the Getting started and Integrations sections as those are more about the product than the platform;

I'm going to keep the getting started section, as information on the product is also integral, but I believe you are right about the integrations section. What I would like eventually is a showcase of our own eventual developer tooling to enable a developer to build their own integrations, but the integrations page is not that.

Split the Community section in two: “Join the community” with a link to the Discord so that folks know that they can reach out to other builders and ask questions, and “Built by the community” with links to open source projects built by the community. Build by the community could be among the first blocks as I find it a very strong signal as a developer, and a good thing to browse and get ideas from.

This is a fabulous idea! I'll integrate it as such.

Change “Beta Features” to “Join the Beta program” to switch from passive voice to the active one. Can also mention that the beta program is a great way to suggest improvements to features/API before they reach GA.

An active positioning means we need to have a link to join. Is it the link to the Airtable form? Ideally, we'd also have a landing page for the beta program that enumerated its benefits that this would link to, instead of directly linking to the form. TBC for later.

That would require deeper changes (although totally feasible) but if we had tags on the blog we could link to our “Engineering” blog directly, i.e. we filter down by the “Engineering” tag.

We do tag blog posts on Sanity. I've been using a "Developer Relations" one for example. Can we filter by those? I'm guessing that data is exposed somehow to our site and we could create a blog view based on tag?

We have a few engineering-focused talks by the Orbit team that could maybe find a home in this page as well? Yours as Rails Conf, mines at the Eleventies and Jamstack Berlin… Not 100% sure this is a good idea, just throwing it out there.

Yes! An entire area devoted to our presence in communities (meetups, talks, etc.) is a must-have. Consider that a coming soon feature.
Design

Wondering whether we should remove the Orbit Model banner at the bottom as it’s standing out a bit too much currently

I was feeling the same way. I do like having some call out to the conceptual work behind Orbit. I'd like for it to be minimal though as a developer page should focus on the practical, nonetheless, a small mention somewhere if someone wanted to read up on it would be good. Can be incorporated later.

I riffed on an alternative to the content on each card:

Much better! 👏🏻

Love how it turned out @bencgreenberg! Thanks for the amazingly quick turnaround on the suggestions. Couple more ideas:

For links inside card subheaders, use underline instead of font-purple-400 so that they feel “less important” than the subheaders links (top is my suggestion, bottom is the current look & feel):

The current contrast ratio of card description falls short of being accessible given the background color. Changing from font-gray-500 to font-gray-600 does the trick. In the following screenshots I change the “_ Explore and utilize open-source tooling built_” bit.

Before:

After:

Here’s a video on how I tested the contrast ratio using the dev tools
  https://user-images.githubusercontent.com/2587348/115002104-a5e4ea00-9ea4-11eb-8957-a90b81754409.mp4

Should Getting Started > Integrations be renamed Getting Started > GitHub Integration?
Orbit API > Batch/Bulk Data Import is also in the Community Built section, we might want to only have it there
Node-RED could link to https://nodered.org/
I think Kevin built another open source project for in-person events, should we add it in the Community Built section?
In the Beta section, Alerts & Destinations and Webhooks are nearly the same thing (i.e. Webhooks is the big thing powered by Alerts & Destinations). We might want to only keep Webhook for conciseness.
Should we add Community Built to the Join the Community card?

Can we filter by those? I'm guessing that data is exposed somehow to our site and we could create a blog view based on tag?

I think everything is set up in Sanity, but we’re missing the filtering code in the website. As it’s a fairly common use-case for Eleventy I don’t anticipate it being that difficult to implement though.

Once we’re confident in the content and the design, there’s a code refactoring that I have in mind that might help create a reusable component for those cards and make future changes simpler. We can leverage Nunjucks’ macros to create a card macro which will take its content as parameters. We’re using that patterns in a few places, most notably in the integrations page.
Here we create a new integration macro:
https://github.com/orbit-love/orbit-web/blob/d656aa888aa6fea18bfe352df5587d11fcf630dc/src/includes/macros/features.njk#L92-L124
Here we instantiate it for e.g. the Discourse integration:
https://github.com/orbit-love/orbit-web/blob/d656aa888aa6fea18bfe352df5587d11fcf630dc/src/integrations.html#L45-L67
Happy to pair on this if you’d like!

But as the website is going to change, I’m not sure yet if that’s worth the work. We can talk about that with Gareth!

That's a very good point. We can hold off.

Once we’re confident in the content and the design, there’s a code refactoring that I have in mind that might help create a reusable component for those cards and make future changes simpler. We can leverage Nunjucks’ macros to create a card macro which will take its content as parameters. We’re using that patterns in a few places, most notably in the integrations page.
Here we create a new integration macro:
https://github.com/orbit-love/orbit-web/blob/d656aa888aa6fea18bfe352df5587d11fcf630dc/src/includes/macros/features.njk#L92-L124
Here we instantiate it for e.g. the Discourse integration:
https://github.com/orbit-love/orbit-web/blob/d656aa888aa6fea18bfe352df5587d11fcf630dc/src/integrations.html#L45-L67
Happy to pair on this if you’d like!

Ah, I just saw this comment now. Yes, a reusable component looks like a good idea. The templating looks similar to Rails templates. I think I can work on this.

I think I can work on this.

Awesome. Let me know if I can help!
Out of curiosity I tried swapping the third and fourth cards, and it mitigates a bit the whitespace caused by the long “Community Built” card. Let me know what you think!

Apart from the few tiny comments/typos, I think this is ready to merge! Feel free to hit that button when you’re ready, it should be deployed about a minute after


cli tests added

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+28.9%) to 81.837%

Files with Coverage Reduction
New Missed Lines
%

cli/pdb4amber.py
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
28.9%

Covered Lines:
802

Relevant Lines:
980

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+28.9%) to 81.837%

Files with Coverage Reduction
New Missed Lines
%

cli/pdb4amber.py
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
28.9%

Covered Lines:
802

Relevant Lines:
980

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+28.9%) to 81.837%

Files with Coverage Reduction
New Missed Lines
%

cli/pdb4amber.py
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
28.9%

Covered Lines:
802

Relevant Lines:
980

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+28.9%) to 81.837%

Files with Coverage Reduction
New Missed Lines
%

cli/pdb4amber.py
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
28.9%

Covered Lines:
802

Relevant Lines:
980

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+28.9%) to 81.837%

Files with Coverage Reduction
New Missed Lines
%

cli/pdb4amber.py
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
28.9%

Covered Lines:
802

Relevant Lines:
980

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+28.9%) to 81.837%

Files with Coverage Reduction
New Missed Lines
%

cli/pdb4amber.py
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
28.9%

Covered Lines:
802

Relevant Lines:
980

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+28.9%) to 81.837%

Files with Coverage Reduction
New Missed Lines
%

cli/pdb4amber.py
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
28.9%

Covered Lines:
802

Relevant Lines:
980

💛  - Coveralls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

0 of 0   changed or added relevant lines in 0 files are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage increased (+28.9%) to 81.837%

Files with Coverage Reduction
New Missed Lines
%

cli/pdb4amber.py
5
80.0%

Totals

Change from base Build<PHONE_NUMBER>:
28.9%

Covered Lines:
802

Relevant Lines:
980

💛  - Coveralls

After all these failed checks, I've had to add to the docs conf.py to ignore built-in warnings for py:Class QbFields, which is a class in aiida


CCSCNE Undergraduate Research Competition

Description
CCSCNE is hosting an undergraduate research competition and we would like to submit something to this competition.
Acceptance Criteria

[ ] Write and submit an abstract

[ ] Write an abstract
[ ] Receive feedback/review of abstract
[ ] Submit by March 5

[ ] Create a poster

[ ] Look at the old poster(s)
[ ] Look at the templates provided
[ ] Create the poster
[ ] Receive feedback/reviews on the poster
[ ] Submit by March 26

[ ] Prep for the in-person presentation

Dates

Abstract deadline - March 6
Poster notification - March 19
Poster due date - March 26

Resources
CCSCNE Undergraduate Research Website

Note: Jaden and Heather will meet on 3/3 to discuss getting started


Integration tests for multiple software pipelines

Develop CI tests that download latest versions of tools and check their input/outputs are AIRR-compliant. This is a bit related to software WG's sumrep bc it'll probably require checking that the actual results are sane too. @scharch

What's the thought here? Output should be easy enough using the validation function in the reference implementation, right?

There are many different projects all being developed separately. The idea is to try to make a system that will alert people when someone breaks something. That means either the standard changes and now everyone needs to be notified. Or a tool makes a change that brings things out of compliance.

In a datarep call we discussed how checking that all tools are AIRR-compliant is a heavy lift, and we are going to let tools self-validate.


Revert "[PDI-16893] TextFileOutput has a dangerous "feature" allowing data to be piped to OS commands."

Reverts pentaho/pentaho-kettle#4944

The Copyright stuff is actually correct because the revert is putting the files back to what they were prior to Aliaksandr's commit


REFACTOR-#5330: remove BaseIO._read

Signed-off-by: Myachev<EMAIL_ADDRESS>
What do these changes do?

We already have _read methods in the dispatcher hierarchy. Since there is no need to make such a method also in the IO hierarchy, we can delete it and stop finding out the method of which of the hierarchies was called.

[x] first commit message and PR title follow format outlined here

NOTE:  If you edit the PR title to match this format, you need to add another commit (even if it's empty) or amend your last commit for the CI job that checks the PR title to pick up the new PR title.

[x] passes flake8 modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] passes black --check modin/ asv_bench/benchmarks scripts/doc_checker.py
[x] signed commit with git commit -s
[x] Resolves #5330
[ ] tests added and passing
[x] module layout described at docs/development/architecture.rst is up-to-date

Blocked on https://github.com/modin-project/modin/pull/5328

@modin-project/modin-hdk need approval please. @vnlitvinov ?


updating kafka avro and delete consumer docs

Moved some IDK content around. Made the kafka ingest page the kafka-static ingest page. Then moved CSV and SQL content on that page to the Ingesters page (just a lazy way to make sure I'm not removing content).
Then created an avro and avro delete page with updated content from work done on FB-1843 and FB-1846.

@jrbrinlee1 thank you for this! Is there a current need to get this in? @ljsinclair has changes for kafka that is probably best to have her absorb in.

Hi there,
I'd say it's ok to merge these changes in as-is.
I'll add them into the branch I'm working on to redesign the kafka ingest docs -- com-ingest-kafka

Thanks @gthrone25 and @ljsinclair for taking a look and sorry for the delay. Merging these in but let me know you have any questions on the content once you start making some changes.


[BEAM-7923] Emit info when capture control options are configured.

Info logging to notify users about invoking evict_captured_data on
changing capture_duration, capture_size_limit, and capturable_sources
options. The users have to explicitly evict captured data because the
process is destructive and the usage should be the same as evicting
captured data even when they don't modify options at all.
Info logging about behavior difference between enabling capture
replay and disabling it. This functions as a switch enabling/disabling
the replayability, thus works without the need of evicting captured
data.
Changed the default capture_duration from 5s to 60s.

Please add a meaningful description for your change here

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Choose reviewer(s) and mention them in a comment (R: @username).
[x] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] Update CHANGES.md with noteworthy changes.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Per discussion offline, we've decided to emit some logging for options APIs.
yapf formatted.
lint passed locally.
R: @pabloem
R: @davidyan74
R: @rohdesamuel

retest this please

retest this please

retest this please


scipy.fft.next_fast_len() should be backend-aware

This issue stems from a discussion in the downstream: https://github.com/cupy/cupy/pull/3560#issuecomment-655204902. I don't have a bug reproducer for the moment, but I think the below description should make it clear that it's a potential bug to be addressed.
With the recent backend support in scipy.fft, perhaps scipy.fft.next_fast_len() should also be registered with a backend? Ideally, we want this usage
import scipy.fft as spfft
import cupyx.scipy.fft as cufft

spfft.set_global_backend(cufft)
n = spfft.next_fast_len(100)

to correctly return the number n for cuFFT (CUDA's FFT library), not for pocketfft. In other words, every downstream library that supports SciPy FFT's backend mechanism is required to implement its own next_fast_len().
cc: @grlee77 @peterbell10

I don't have a bug reproducer for the moment

To clarify, it is not an issue of potentially incorrect results for the FFT, only that the length recommended by the SciPy implementation could be suboptimal in terms of computation time. The underlying FFT implementations in cuFFT and pyFFTW, for example, are optimized for a slightly different set of small prime factors than pocketfft. Specifically there is no special implementation for prime factors of 11 in cuFFT, but SciPy's complex-to-complex transforms have one.

I can see two issues with this:

next_fast_len doesn't take any array arguments so if you have multiple active backends (say CuPy and pyFFTW) then it might pick the wrong version.
We use caching so next_fast_len is really fast to call multiple times, only 50 ns. If we put the cache behind a dispatch layer it grows to ~500 ns.

One solution could be to add a pad_to_fast_length=True (or something similar) option to the transforms. The implementation can call their own version of next_fast_len internally. They call it directly so there is no overhead, plus it may actually make the caller's code simpler.

next_fast_len doesn't take any array arguments so if you have multiple active backends (say CuPy and pyFFTW) then it might pick the wrong version.

IIUC we could add an xp kwarg (which is consistent with how we've treated the array creation functions (r)fftfreq) to next_fast_len and uarray could determine the backend from xp if it is set.


Adds dockerignore file to save space

Adds .dockerignore to avoid copying in .git into containers.

@vpetersson do we need to add this for UPnP, miner, pktfwd as well?

Turns out this isn't actually needed for .git on balena...
https://www.balena.io/docs/learn/develop/dockerfile/#using-dockerfiles-with-balena

Presumably this is only true if you're building the images on Balena. We're not. We're building using docker buildx and are then pushing to Docker Hub. These images are then consumed by Balena. Hence .git is already part of the image (unless excluded) by the time it is pulled down by Balena.


⚠ ESPN has degraded performance

In 7ce45c1, ESPN (https://espn.com) experienced degraded performance:

HTTP code: 200
Response time: 1086 ms

Resolved: ESPN performance has improved in 752e75d.


Logo and readme

PR Checklist

[x] Please add a description of your changes.
[x] We need your changes to come with unit tests in order to keep this project in quality and easy to maintain.
[] Please add a link to the open issue or task that this pull request will resolve.
[ ] Add a note to the changelog to indicate the change.

Does this PR introduce a breaking change?

[ ] Yes
[x] No

If yes please indicate it and migration info into the CHANGELOG.md file.

This PR added the logo to our readme and several badges about sonar ( quality gate is unfortunately in fail state but I am sure will be soon green). I added a list of our main pillars and reason to choose this project

@jerome-benoit  @Shinigami92  behind the PR I would like to have feedbacks on our list of "why use poolifier", please suggest any further feature/highlight that we can add in that list or even a change with another already there.

@pioardi

You should center the logo
The logo is a bit to big in size
You can also center the badges too
Behind security the two badges need a space in between

No runtime dependencies white_check_mark (until now we don't have any exception to that) could be better No runtime dependencies :heavy_check_mark: (until now we don't have any expectations to add some)
Active community -> "There are three active maintainers" or similar
Maybe the whole list needs a better representation / design? Maybe a table or so?!

@ST-DDT Do you also want to have a look as an outside person?

File can be viewed rawly here: https://github.com/pioardi/poolifier/blob/logo-and-readme/README.md

@pioardi

You should center the logo
The logo is a bit to big in size
You can also center the badges too
Behind security the two badges need a space in between

No runtime dependencies white_check_mark (until now we don't have any exception to that) could be better No runtime dependencies :heavy_check_mark: (until now we don't have any expectations to add some)
Active community -> "There are three active maintainers" or similar
Maybe the whole list needs a better representation / design? Maybe a table or so?!

@ST-DDT Do you also want to have a look as an outside person?

I decreased logo size and centered the logo
I change also the point on runtime dependencies and added a space between the two security badges
On active community we really don't need to say how much members are into it, we are an active community anyway
I would need your help to center badges and find a better representation for the list

You need html as a fallback for that
You can look into my plugin https://raw.githubusercontent.com/prettier/plugin-pug/main/README.md
The logo now has a good size :+1:

The centered logo looks good to me. And I would probably also center the badges and the title (not sure whether a centered title looks good/intuitive, but badge and title alignment should be the same).
However I would reorder the badges to match their group/importance/relevance for external users. Suggestion:

Downloads
NodeCI
Coverage
QualityGamte
Gitter
PRswelcome
Deps+Deps Bot

@pioardi looking MUCH better now

@pioardi looking MUCH better now

Ohhh yeah bro, this is the result when we use feedback loops :) It is one of my main mantra also during paid job

@Shinigami92  if you approv, I guess we are ready to merge this

Thanks also to @ST-DDT, I will include you soon into our readme as PR reviewer(together with some other occasional contributors)


Static HTML output

Is it needed and if yes, in what form??
I could imagine a directory (A-Z) with static game entries. But I'm not sure if this is really needed. Can we get the dynamic table to do that?

Either we let the dynamic table to all the work (will require extending it) or we let static html do all the work (will result in many static files with similar content). Not sure what is better. Will evaluate it more.

In any case the link target for all repositories of form https://anongit.kde.org/XXX.git should be https://cgit.kde.org/XXX.git because that gives a better web view. It's not necessary for Github, Gitlab, ...

The same goes for Sourceforge. The git at https://git.code.sf.net/p/XXX/YYY has a web view of form https://sourceforge.net/p/XXX/YYY/ci/master/tree/

SVNa at http://svn.code.sf.net/p/XXX/YYY/ have a web view of the form https://sourceforge.net/p/XXX/YYY/HEAD/tree/

See also #213

https://bitbucket.org/XX/YY.git have their webview (and redirect to) https://bitbucket.org/XX/YY/commits/

I decided on a combination of both, a more extended static website including author and originals but also with dynamic tables.
Technically, I will use a template engine (Jinja2), a CSS framework and Javascript dynamic tables to build a static website for the entries in this database.
Now the task is to design the static website.

I'm quite advanced.

The static HTML output is live, just needs some tweaks in the layout and text on the contribute page and statistics, otherwise it's fine.  Maybe a dynamic HTML table for quick searching would be nice again.


How to use a translation in an alt-Attribute of an img tag?

I tried the following:
<img src='...' alt="{{'NS_ELEMENTS.HELP' | translate}}" />

is not working and an ng-alt is not existing.
I've ended up with getElementById and manipulating the dom directly,
but this isn't working, when the language is changed dynamically.
Any Help?
Thanks in advance

Well it works in general?
See here: http://plnkr.co/edit/PFn2DAsR3D6Z52gz7lbn?p=preview

You're right. I made a mistake in the language files. Now it is working.


fix: Add chain fallback for wallet switching

Problem solved
Short description of the bug fixed or feature added

PR-Codex overview
This PR addresses an issue where adding a blockchain network was not triggering for certain wallets. It modifies the error handling in the wallet_addEthereumChain process to ensure the chain is added correctly when the error code indicates it does not exist.
Detailed summary

Removed explicit any type handling in the catch block.
Simplified error handling by removing the conditional check for error codes.
Ensured the addition of the chain is attempted unconditionally when an error occurs.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

#5668  👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.


Project Status

Hello, I'm looking to manage as much of our infrastructure as possible from Crossplane.
To do so I would like to manage Vault policies and roles as well.
As far as I can tell this project is basically unused and unmaintained. Is that the case?
Further, the APIs do not seem to reflect even a fraction of the Terraform Vault provider's capabilities.
https://registry.terraform.io/providers/hashicorp/vault/latest/docs

I am currently just exploring using the Terraform Crossplane Provider directly for my use case, but if there's support I'd be interested in contributing to a full Vault provider.
Thank you!

@reedjosh this provider has been superceded by https://github.com/upbound/provider-vault


[Feature] Add Module Linkis MetaData

Search before asking

[X] I had searched in the issues and found no similar feature requirement.

Problem Description
Supply Linkis Metadata Service
Description
Add Module linkis-metadata
Use case
No response
solutions
No response
Anything else
No response
Are you willing to submit a PR?

[X] Yes I am willing to submit a PR!

The issue has finished


复杂网络常用数据集网站

作者：Hdevin
来源：http://www.jianshu.com/p/9313bc75c94b
2016.09.21 08:36
复杂网络的研究很多都离不开数据集，下面这些是个人在做科研的过程中在互联网上搜集到的一些数据集网站，列举出来也方便同行们去使用。
1、http://vladowiki.fmf.uni-lj.si/doku.php?id=pajek:data:urls:index
数据集网站集合。这个网站中列出来很多数据集网站，非常全，其中的很多网站数据描述详细，而且数据可以直接下载，大家可以从中选出一些网站收藏起来。
2、http://snap.stanford.edu/data/
这个是斯坦福大学的大型网络数据集网站，大家应该比较熟悉了。
3、http://konect.uni-koblenz.de/
这个网站是我个人最喜欢也是最常用的网站，里面数据集有上百种，数据的分类和描述真的是特别详细，而且还给出了数据集的可视化图以及一些基本统计特性，所有数据均可以直接下载。
4、http://networkrepository.com/index.php
这是另一个个人特别喜欢也是很常用的网站，和上面一样数据分类也是相当的详细，大家应该都能找到自己想要的数据。
5、http://gdm.fudan.edu.cn/GDMWiki/Wiki.jsp?page=Network DataSet
这是复旦大学建立的网络数据集网站，里面有一些常用数据集以及一些相关资源网站。
6、https://toreopsahl.com/datasets/
这个网站里面有十几个数据集，包含社交网络、交通网络、合作网络等常用数据集。
7、http://netwiki.amath.unc.edu/SharedData/SharedData
这个网站列出了一些数据集以及一些复杂网络大牛的个人数据网站，大家可以去看看。
8、http://vlado.fmf.uni-lj.si/pub/networks/data/
这个是Pajek网站提供的数据集，里面的数据都很经典，复杂网络早期研究中很多数据集都是源于这里。
9、http://socialcomputing.asu.edu/pages/datasets
社交网络数据集，里面包含国内外一些常用在线社区网站的数据集，中型到大型的数据规模，搞社会计算的同行可能用的比较多。
10、http://www.sociopatterns.org/datasets/
另一个社交网络数据集，这里面的数据集更多倾向于实体网络，比如联系网、高校朋友网、疾病传播网。
11、http://www3.nd.edu/~networks/resources.htm
CCNR，大牛Barabási教授个人的数据网站，当然了除开数据集，这个网站上有很多可以学习的东西。
12、http://www-personal.umich.edu/~mejn/netdata/
大牛Newman教授的个人数据网站，里面的数据集特别是科学家合作网络，用到特别多。
以上是一些个人用到比较多的复杂网络数据网站，鉴于自己的了解有限，可能对这些数据集网站给出的描述还不够详细，而且很多单位都没有列出来，敬请谅解。当然了如果想用里面的数据集，一定要记得引用数据集作者们的信息，他们收集数据并公开方便大家使用也是很不容易的。如果转载本文，也请注明出处^_^.
<EMAIL_ADDRESS>

title: "互联网行为数据列表"
date: 2013-12-15 20:24
comments: true
categories:

python
tags:
chengjun
lingfei
数据
互联网

Publicly Accessible Datasets of Human Online Behavior
Edited by Lingfei Wu & Chengjun Wang Sep.15, 2011
Refer to google docs
New Added Data are listed below:

2012

ICWSM 2011 Spinn3r Dataset
http://www.icwsm.org/data/
That dataset, provided by Spinn3r.com, is a continuation of the 2009 Spinn3r Dataset. The dataset consists of over 386 million blog posts, news articles, classifieds, forum posts and social media content between January 13th and February 14th. It spans events such as the Tunisian revolution and the Egyptian protests (see http://en.wikipedia.org/wiki/January_2011 for a more detailed list of events spanning the dataset’s time period).
1.Reddit Voting data
http://www.reddit.com/r/redditdev/comments/lowwf/attempt_2_want_to_help_reddit_build_a_recommender/

Tencent weibo mining

http://www.kddcup2012.org/c/kddcup2012-track1/data

Rating Analysis and online forum mining of apple

http://sifaka.cs.uiuc.edu/~wang296/Data/index.html

Web Research Collections of blog and government website

http://ir.dcs.gla.ac.uk/test_collections/

Query Data of Microsoft Research

QRU-1: A Public Dataset for Promoting Query Representation and Understanding Research.
Data Set: http://research.microsoft.com/en-us/downloads/d6e8c8f2-721f-4222-81fa-4251b6c33752/
Paper: http://research.microsoft.com/en-us/people/hangli/qru-1.pdf
A new public dataset for promoting query representation and understanding research, referred to as QRU-1, was recently released by Microsoft Research. The QRU-1 dataset contains reformulations of Web TREC topics that are automatically generated using a large-scale proprietary web search log, without compromising user privacy. In this paper, we describe the content of this dataset and the process of its creation. We also discuss the potential uses of the dataset, including a detailed description of a query reformulation experiment.

2011 & Before

Project 1. Stanford Large Network Dataset Collection
http://snap.stanford.edu/data/index.html
● Socialnetworks: online social networks, edges represent interactions between people
● Communicationnetworks: email communication networks with edges representing communication
● Citationnetworks: nodes represent papers, edges represent citations
● Collaborationnetworks: nodes represent scientists, edges represent collaborations (co-authoring a paper)
● Webgraphs: nodes represent webpages and edges are hyperlinks
● Amazonnetworks : nodes represent products and edges link commonly co-purchased products
● Internetnetworks : nodes represent computers and edges communication
● Roadnetworks : nodes represent intersections and edges roads connecting the intersections
● Autonomoussystems : graphs of the internet
● Signednetworks : networks with positive and negative edges (friend/foe, trust/distrust)
● Wikipedianetworksandmetadata : Talk, editing and voting data from Wikipedia
● TwitterandMemetracker : Memetracker phrases, links and 467 million Tweets
Social networks
Name	Type	Nodes	Edges	Description
soc-Epinions1	Directed	75,879	508,837	Who-trusts-whom network of Epinions.com
soc-LiveJournal1	Directed	4,847,571	6,8993,773	LiveJournal online social network
soc-Slashdot0811	Directed	77,360	905,468	Slashdot social network from November 2008
soc-Slashdot0922	Directed	82,168	948,464	Slashdot social network from February 2009
wiki-Vote	Directed	7,115	103,689	Wikipedia who-votes-on-whom network
Communication networks
Name	Type	Nodes	Edges	Description
email-EuAll	Directed	265,214	420,045	Email network from a EU research institution
email-Enron	Undirected	36,692	367,662	Email communication network from Enron
wiki-Talk	Directed	2,394,385	5,021,410	Wikipedia talk (communication) network
Citation networks
Name	Type	Nodes	Edges	Description
cit-HepPh	Directed, Temporal, Labeled	34,546	421,578	Arxiv High Energy Physics paper citation network
cit-HepTh	Directed, Temporal, Labeled	27,770	352,807	Arxiv High Energy Physics paper citation network
cit-Patents	Directed, Temporal, Labeled	3,774,768	16,518,948	Citation network among US Patents
Collaboration networks
Name	Type	Nodes	Edges	Description
ca-AstroPh	Undirected	18,772	396,160	Collaboration network of Arxiv Astro Physics
ca-CondMat	Undirected	23,133	186,936	Collaboration network of Arxiv Condensed Matter
ca-GrQc	Undirected	5,242	28,980	Collaboration network of Arxiv General Relativity
ca-HepPh	gUndirected	12,008	237,010	Collaboration network of Arxiv High Energy Physics
ca-HepTh	Undirected	9,877	51,971	Collaboration network of Arxiv High Energy Physics Theory
Web graphs
Name	Type	Nodes	Edges	Description
web-BerkStan	Directed	685,230	7,600,595	Web graph of Berkeley and Stanford
web-Google	Directed	875,713	5,105,039	Web graph from Google
web-NotreDame	Directed	325,729	1,497,134	Web graph of Notre Dame
web-Stanford	Directed	281,903	2,312,497	Web graph of Stanford.edu
Product co-purchasing networks
Name	Type	Nodes	Edges	Description
amazon0302	Directed	262,111	1,234,877	Amazon product co-purchasing network from March 2 2003
amazon0312	Directed	400,727	3,200,440	Amazon product co-purchasing network from March 12 2003
amazon0505	Directed	410,236	3,356,824	Amazon product co-purchasing network from May 5 2003
amazon0601	Directed	403,394	3,387,388	Amazon product co-purchasing network from June 1 2003
amazon-meta	Metadata	548,552	1,788,725	Amazon product metadata: product info and all reviews on around 548,552 products.
Internet peer-to-peer networks
Name	Type	Nodes	Edges	Description
p2p-Gnutella04	Directed	10,876	39,994	Gnutella peer to peer network from August 4 2002
p2p-Gnutella05	Directed	8,846	31,839	Gnutella peer to peer network from August 5 2002
p2p-Gnutella06	Directed	8,717	31,525	Gnutella peer to peer network from August 6 2002
p2p-Gnutella08	Directed	6,301	20,777	Gnutella peer to peer network from August 8 2002
p2p-Gnutella09	Directed	8,114	26,013	Gnutella peer to peer network from August 9 2002
p2p-Gnutella24	Directed	26,518	65,369	Gnutella peer to peer network from August 24 2002
p2p-Gnutella25	Directed	22,687	54,705	Gnutella peer to peer network from August 25 2002
p2p-Gnutella30	Directed	36,682	88,328	Gnutella peer to peer network from August 30 2002
p2p-Gnutella31	Directed	62,586	147,892	Gnutella peer to peer network from August 31 2002
Road networks
Name	Type	Nodes	Edges	Description
roadNet-CA	Undirected	1,965,206	5,533,214	Road network of California
roadNet-PA	Undirected	1,088,092	3,083,796	Road network of Pennsylvania
roadNet-TX	Undirected	1,379,917	3,843,320	Road network of Texas
Autonomous systems graphs
Name	Type	Nodes	Edges	Description
as-733(733 graphs)	Undirected	103-6,474	243-13,233	733 daily instances(graphs) from November 8 1997 to January 2 2000
as-Skitter	Undirected	1,696,415	11,095,298	Internet topology graph, from traceroutes run daily in 2005
as-Caida (122 graphs)	Directed	8,020-26,475	36,406-106,762	The CAIDA AS Relationships Datasets, from January 2004 to November 2007
Oregon-1(9 graphs)	Undirected	10,670-11,174	22,002-23,409	AS peering information inferred from Oregon route-views between March 31 and May 26 2001
Oregon-2 (9 graphs)	Undirected	10,900-11,461	31,180-32,730	AS peering information inferred from Oregon route-views between March 31 and May 26 2001
Signed networks
Name	Type	Nodes	Edges	Description
soc-sign-epinions	Directed	131,828	841,372	Epinions signed social network
wiki-Elec	Directed, Bipartite	~7,000	~100,000	Wikipedia adminship election data
soc-sign-Slashdot081106	Directed	77,357	516,575	Slashdot Zoo signed social network from November 6 2008
soc-sign-Slashdot090216	Directed	81,871	545,671	Slashdot Zoo signed social network from February 16 2009
soc-sign-Slashdot090221	Directed	82,144	549,202	Slashdot Zoo signed social network from February 21 2009
Wikipedia networks and metadata
Name	Type	Nodes	Edges	Description
wiki-Vote	Directed	7,115	103,689	Wikipedia who-votes-on-whom network
wiki-Talk	Directed	2,394,385	5,021,410	Wikipedia talk (communication) network
wiki-Elec	Bipartite	~7,000	~100,000	Wikipedia adminship election data
wiki-meta	Edits	2.3M users,
3.5M pages	250M edits	Complete Wikipedia edit history (who edited what page)
Memetracker and Twitter
Name	Type	Nodes	Edges	Description
twitter7	Tweets	17,069,982 users	476,553,560 tweets	A collection of 476 million tweets collected between June-Dec 2009
memetracker9	Memes	96 million	418 million links	Memetracker phrases and hyperlinks between 96 million blog posts from Aug 2008 to Apr 2009
ksc-time-series	Time
Series	2,000	418 million links	Time series of volume of 1,000 most popular Memetracker phrases and 1,000 most popular Twitter hashtags
Project 2. MemeTracker
http://www.memetracker.org/data.html
MemeTracker builds maps of the daily news cycle by analyzing around 900,000 news stories and blog posts per day from 1 million online sources, ranging from mass media to personal blogs. We track the quotes and phrases that appear most frequently over time across this entire online news spectrum. This makes it possible to see how different stories compete for news and blog coverage each day, and how certain stories persist while others fade quickly. Overall we track more than 17 million different phrases and about 54% of the total phrase/quote mentions appear on blos and 46% in news media.
Dataset size: 220M (clustered), 13.3 G (Zipped raw data over 9 months)
Example of a record in the file: lines below map to the fields above. First line is record A, followed by B and 3 C records. Then another B and 2 C records.
2 8 we’re not commenting on that story i’m afraid 2131865
3 3 we’re not commenting on that 489007
2008-08-18 14:23:05 1 M http://business.theage.com.au/business/bb-chief-set-to-walk-plank-20080818-3xp7.html
2008-11-26 01:27:13 1 B http://sfweekly.com/2008-11-26/news/buy-line
2008-11-27 18:55:30 1 B http://aconstantineblacklist.blogspot.com/2008/11/re-researcher-matt-janovic.html
5 2 we’re not commenting on that story 2131864
2008-12-08 14:50:18 3 B http://videogaming247.com/2008/12/08/home-in-10-days-were-not-commenting-on-that-story-says-scee
2008-12-08 19:35:31 2 B http://jplaystation.com/2008/12/08/home-in-10-days-were-not-commenting-on-that-story-says-scee
Project 3. PINTS – Experiments Data Sets
http://www.uni-koblenz-landau.de/koblenz/fb4/AGStaab/Research/DataSets/PINTSExperimentsDataSets/index_html
Dataset	Users	Tags	Resources	Tag assignm.	Download
Flickr	319,686	1,607,879	28,153,045	112,900,000	flickr_UsrResTag.7z
(518 MB) packed with 7zip
Delicious	532,924	2,481,698	17,262,480	140,126,586	delicious_UsrResTag.7z
(848 MB) packed with 7zip
Project 4. K. Lerman’s datasets
http://www.isi.edu/integration/people/lerman/downloads.html
Digg 2009 This anonymized data set consists of the voting records for 3553 stories promoted to the front page over a period of a month in 2009. The voting record for each story contains id of the voter and time stamp of the vote. In addition, data about friendship links of voters was collected from Digg.
Flickr personal taxonomies This anonymized data set contains personal taxonomies constructed by 7,000+ Flickr users to organize their photos, as well as the tags they associated with the photos. Personal taxonomies are shallow hierarchies (trees) containing collections and their constituent sets (aka photo-albums) and collections.
Wrapper maintenance Wrappers facilitate access to Web-based information sources by providing a uniform querying and data extraction capability. When wrapper stops working due to changed in the layout of web pages, our task is to automatically reinduce the wrapper. The data sets used for experiments in our JAIR 2003 paper contain web pages downloaded from two dozen sources over a period of a year.
Project 5. Facebook datasets
http://odysseas.calit2.uci.edu/doku.php/public:online_social_networks#available_datasets
Facebook social graph
The following datasets are collected in April of 2009 through data scraping from Facebook :

MHRW – A sample of 957K unique users obtained Facebook-wide by 28 independent Metropolis-Hastings random walks

UNI – A sample of 984K unique users that represents the “ground truth” i.e., a truly uniform sample of Facebook userIDs, selected by a rejection sampling procedure from the system’s 32-bit ID space.

UserIDs are consistent across files. UserIDs and networkIDs are anonymized. The mapping for networkIDs is available upon request.
Facebook applications
Dataset I contains the number of active users and total application installations daily for every Facebook application between 08/29/2007 and 02/14/2008 . The data was retrieved from the Adonomics website, which had been collecting aggregate applications statistics, Daily Active Users (DAU) and Application Installs, by scraping the Facebook application directory.
Dataset I comprises of 16,812 files (one file for each application present in the Facebook application directory until 02/14/2008).
Dataset II is collected in February 2008 and contains a list of installed applications for 297K Facebook users.
Facebook weighted random walks
The following datasets is collected in October of 2010 through data scraping from Facebook :
RW – A sample of 1M unique users obtained Facebook-wide by 25 independent simple Random Walks
Hybrid – A sample of 1M unique users obtained Facebook-wide by 25 independent Stratified Weighted Random Walks (S-WRW) with hybrid conflict resolution. The measurement objective in the Hybrid sample are Facebook users with college network membership.
For each dataset, we release two files. The first file contains for each sampled userID, (i) the weight of the sampled user, (ii) the number of vfriends, the visitable friends during the social graph exploration (or friends for which “View Friends”=1), (iii) the total number of friends , and (iv) list of networkIDs of which the user is a member of.
Project 6. Network data released by Eric D. Kolaczyk
http://math.bu.edu/people/kolaczyk/datasets.html
A list of datasets used in the book Statistical Analysis of Network Data are provided. For each of those datasets available, the author has combined a data file(s) with a README file, in the format of a compressed ZIP file. In the README file are given a description of the data, a brief characterization of the context in which they arise, and relevant information on their source.
3 of all the datasets concenring human online behavior are worth paying attention to.
AIDS blogs: Network of citations among blogs related to AIDS, patients, and their support networks, collected by Gopal, over a three-day period in August 2005.
Packet delay: Packet delay data from Coates et al. resulting from an Internet packet probing experiment designed for conducting network topology inference.
Router-level Internet: A network representation of a portion of the router-level Internet, based on topology discovery measurements collected between April 21 and May 8, 2003 by the skitter measurement system atCAIDA.
Project 7. The Internet Radar Datasets
http://www-rp.lip6.fr/~latapy/Radar/
The authors design and implement an ego-centered measurement tool, and perform radar-like measurements consisting of repeated measurements of the internet topology. They conduct long-term (several weeks) and highspeed (one round every few minutes) measurements of this kind from more than one hundred monitors, and provide the obtained data, including the time-variant traffic and topology datasets.
Project 8. The eDonkey Datasets
http://www-rp.lip6.fr/~latapy/P2P_data/
The authors presents a capture of the queries managed by an eDonkey server during almost 10 weeks, leading to the observation of almost 9 billion messages involving almost 90 million users and more than 275 million distinct files.
The data is splitted into one directory per week, one subdirectory per day, one subsubdirectory by hour, and in each of these subsubdirectories three files:
FileSearch.xml.gz contains file searches, i.e. queries based on keywords and metadata sent by clients, and answers (lists of FileId, filenames and metadata) from the server;
SourceSearch.xml.gz contains source searches, i.e. queries send by clients to find providers for given FileId, and answers (lists of providers) from the server;
Main.xml.gz contains basically all other queries, sent to the server to know its load (number of clients and files), to get a textual description of the server, or to get the list of other servers the server knows, and corresponding answers from the server. Notice that the server may also send this kind of queries to other servers, which we then store in this file.
Project 9. The books and music on Amazon
http://www.lambiotte.be/data.html
The author collect the time-variant ranking record of 111 music and books on Amazon and the user data from audioscrobbler, a music library of more than 30000 persons.
Project 10. The University of Florida Sparse Matrix Collection
http://www.cise.ufl.edu/research/sparse/matrices/groups.html
The University of Florida Sparse Matrix Collection is a large and actively growing set of sparse matrices that arise in real applications. The Collection is widely used by the numerical linear algebra community for the development and performance evaluation of sparse matrix algorithms. Its 172 matrices cover a wide spectrum of domains, include those arising from problems with underlying 2D or 3D geometry (as structural engineering, computational fluid dynamics, model reduction, electromagnetics, semiconductor devices, thermodynamics, materials, acoustics, computer graphics/vision, robotics/kinematics, and other discretizations) and those that typically do not have such geometry (optimization, circuit simulation, economic and financial modeling, theoretical and quantum chemistry, chemical process simulation, mathematics and statistics, power networks, and other networks and graphs).
Project 11. Facebook-like social Networks
http://toreopsahl.com/datasets/#online_social_network
The Facebook-like Social Network originate from an online community for students at University of California, Irvine. The dataset includes the users that sent or received at least one message (1,899). A total number of 59,835 online messages were set over 20,296 directed ties among these users. This network has also been described in Patterns and Dynamics of Users’ Behaviour and Interaction: Network Analysis of an Online Community and used in a number of articles including Prominence and control: The weighted rich-club effect and Clusteringinweightednetworks. Although this dataset contains many nodal attributes (e.g., gender, age, and course attended), these are not made available as it would be possible to reverse engineer the anonymisation procedure of users.
Weighted longitudinal one-mode network (weighted by number of characters): tnet-format
Binary longitudinal one-mode network: tnet-format
Weighted static one-mode network (weighted by number of characters): tnet-format; UCINET-format
Weighted static one-mode network (weighted by number of messages): tnet-format; UCINET-format
Project 12. The CAIDA Anonymized 2008-2011 Internet Traces Dataset
http://www.caida.org/data/passive/passive_2011_dataset.xml
This dataset contains anonymized passive traffic traces from CAIDA’s equinix-chicago and equinix-sanjose monitors on OC192 Internet backbone links. This data is useful for research on the characteristics of Internet traffic, including application breakdown, security events, geographic and topological distribution, and flow volume and duration.
The first traffic trace available is a 1 hour traffic trace collected during the DITL 2008 measurement event. This trace contains anonymized packet headers in pcap format on a single direction of the bidirectional OC192 link at equinix-chicago from approximately 2008-03-19 19:00 to 20:00 UTC. The hardware monitoring the other direction of the link was not functioning properly at the time of the traffic capture, so only data for a single direction was captured.
For the equinix-chicago monitor, the first monthly bidirectional traffic trace was taken on April 30 2008, and added to the Anonymized 2008 Internet Trace dataset in June 2008. This 1 hour trace resulted in 83 GB of compressed pcap files. The first monthly bidirectional traffic trace from the equinix-sanjose monitor was taken on July 17 2008.
Traffic traces in this dataset are anonymized using CryptoPAn prefix-preserving anonymization. All traces in this dataset are anonymized with the same key. In addition, the payload has been removed from all packets.
Information on the Anonymized 2009-2011 InternetTracesDatasetcan also be found.
Project 13. Wikipedia Data from Amazon
http://aws.amazon.com/search?searchPath=datasets&searchQuery=wikipedia&x=0&y=0
WikipediaTrafficStatisticsV2
Contains 16 months of hourly pageview statistics for all articles in Wikipedia
WikipediaExtraction (WEX)
A processed dump of the English language Wikipedia
WikipediaPageTrafficStatistics
Contains 7 months of hourly pageview statistics for all articles in Wikipedia
WikipediaPageTrafficStatisticV3
This dataset contains a 150 GB sample of the data used to power trendingtopics.org . It includes a full 3 months of hourly page traffic statistics from Wikipedia (1/1/2011-3/31/2011).
DBpedia 3.5.1
DBpedia is a community effort to extract structured information from Wikipedia and to make this information available on the Web
Wikipedia XML Data
A complete copy of all Wikimedia wikis, in the form of wikitext source and metadata embedded in XML.
Project 14. CiteULike data
http://static.citeulike.org/data/current.bz2
The CiteULike database is potentially useful for researchers in various fields. Physicists and computer scientists have expressed an interest in trying to analyse the structure of the data, and frequently ask for datasets to be made available.
Who-posted-what data
The latest data snapshot can always be downloaded at http://static.citeulike.org/data/current.bz2
Older datasets are available on a daily basis and can be found at URLs of the form http://static.citeulike.org/data/2007-05-30.bz2
Data is available from 2007-05-30 onwards.
The file constitutes an anonymous dump of who posted what and when the posting took place. There is no data in this file which is not already available publicly through the web site, so there are no privacy implications for making it available. The advantage is that it’s available in one file rather than having to spider the entire site to get at the information (please don’t do that!).
The file is a simple unix (“\n” line endings) text file with pipe (“|”) delimiters. The columns are:
The CiteULike article id which was posted
An obfuscated representation of the username (a salted MD5 hash of the true username). Again, it is possible to piece back together what the true username is by scraping the site, but I’d rather you didn’t do that. The reason I’ve gone to the trouble of obfuscation is primarily a slightly paranoid anti-spam measure
The date and time the article was posted to the site
The tag the user used to post it
NB If a user posts an article with n tags, then this will result in n rows in the file
Article linkout data
Mapping CiteULike article_ids to resources on the web can be done with the linkout table. The current snapshot is available at http://static.citeulike.org/data/linkouts.bz2
Older datasets are available on a daily basis and can be found at URLs of the form http://static.citeulike.org/data/linkouts-2008-02-02.bz2
Data is available from 2008-02-02 onwards.
To understand the data in this file, you should refer to “The linkout formatter” section of the plugindeveloper‘sguide.
This file contains a number of spam links. Although CiteULike filters spam postings, traces of the spam still remain in this table. In time this spam content will eventually be removed.
The file is a simple unix (“\n” line endings) text file with pipe (“|”) delimiters. Literal pipes within the fields are represented escaped (“|”). The columns are:
Article Id
Linkout type
ikey_1
ckey_1
ikey_2
ckey_2
NB If an article has n linkouts, then this will result in n rows in the file.
Group membership data
The latest data snapshot can always be downloaded at http://static.citeulike.org/data/groups.bz2
Older datasets are available on a daily basis and can be found at URLs of the form http://static.citeulike.org/data/groups-2008-11-14.bz2
Data is available from 2008-11-14 onwards.
The file constitutes an anonymous dump of who is a member of each group. There is no data in this file which is not already available publicly through the web site, so there are no privacy implications for making it available. The advantage is that it’s available in one file rather than having to spider the entire site to get at the information (please don’t do that!).
The file is a simple unix (“\n” line endings) text file with pipe (“|”) delimiters. The columns are:
An obfuscated group identifier.
An obfuscated representation of the username (a salted MD5 hash of the true username, in the same way as is done for the article posting data).
NB If a group has n members, then this will result in n rows in the file.
Project 15. The Swarmagent Dataset
http://www.swarmagents.com/thesis/detail.asp?id=350
The dataset contains the IDs of users on the brainstorm forum of the Swarmagent Club, the contents of the threads they initiate or reply, and the date and time they do these. The dataset covers the period from the 2003 to Sep. 10, 2010. A readme.txt summarizing the dataset is also provided.
The raw data is in XML format. ACSV version of the dataset can be download from (thanks to Sijie Liu’s efforts) http://www.bjt.name/wp-content/uploads/2010/12/sw_data.rar
Project 16. The Hyperreal User Browsing Dataset
http://www.cs.washington.edu/research/adaptive/download.html
The dataset contains the user logs come from the Music Machines web site at Hyperreal. They have been anonymized (stripped of all information about users except for their succession of accesses to the site).
Each file contains all accesses to Music Machines for a single day. The accesses are organized into paths. Each path is the series of URLs requested from a particular machine. Note that we do not distinguish among multiple users coming from the same source. We have, however, disabled caching of pages at the site so that every page must be requested, even when revisited.
A typical path will appear as below. The first line contains the originating machine (converted to unique numbers for the sake of anonymity). Each succeeding line corresponds to one URL requested from that machine. Each request contains the originating machine (O), the time of the request (T), the URL requested (U), and the referring URL (R). Fields are separated by “||”.
—O:0000002560—
O:0000002560 || T:1997/09/12-22:43:00 || U:/ || R:http://www.hyperreal.org/
O:0000002560 || T:1997/09/12-22:50:27 || U:/categories/software/ || R:http://www.hyperreal.org/music/machines/
O:0000002560 || T:1997/09/12-22:50:38 || U:/categories/software/Windows/ || R:http://www.hyperreal.org/music/machines/categories/software/
O:0000002560 || T:1997/09/12-22:50:47 || U:/categories/software/Windows/V909V03.TXT || R:http://www.hyperreal.org/music/machines/categories/software/Windows/
O:0000002560 || T:1997/09/12-22:51:06 || U:/categories/software/Windows/ || R:http://www.hyperreal.org/music/machines/categories/software/
O:0000002560 || T:1997/09/12-22:51:18 || U:/categories/software/Windows/ravemusc.txt || R:http://www.hyperreal.org/music/machines/categories/software/Windows/
Files are named m.YYMMDD.paths, where YYMMDD represents the date. The authors trained on one month of data and tested on another ten days. The logs from September and October 1997 are presented in one ZIP file per month.
ir.uiowa.edu/polisci_nump/index.2.html
Twitter 2010 data set http://www.isi.edu/integration/people/lerman/load.html?src=http://www.isi.edu/~lerman/downloads/twitter/twitter2010.html


Add web demo link

Hi, I made a web demo for this repo. I would appreciate it if you could add a link to it.

Looks awesome! Thanks so much @hysts!


can you calculate the 'next_version' ?

love the project, but feel like I might be missing something ..
I'd like to be able to print or write the 'next_version'  ( or the next release version )
e.g.
current tag = 0.2.0
distance version on branch (format "{version}.post{distance}") =  0.2.0.post7
next_version (smallest-release) =  0.2.1
how do I print (or write) the 'next version'   ..  for example if I wanted to  tag a repo after merge into the main branch
e.g git tag $(versioningit -n)
I'm happy to open a PR to support this, but wanted to check, am I missing something about how this package should be used ?

While currently (as of v0.2.1), the command-line interface does not support getting the next version, it is possible to compute it using versioningit's Python API.  While the relevant part of the API is documented in the sense of "has docstrings", it's not mentioned in the README, as I think we can both agree the README is long enough as-is.  (If/when this project gets a Readthedocs site, it would be documented there, though.)
However, in response to your post, I've added a get_next_version() function (not listed in the README, though the README now at least points out that there are aspects of the API not listed there) and an -n/--next-version CLI flag for getting the next version of a project.  These features are currently available on the project's master branch, and, if you are satisfied with them, I will release v0.3.0 with them soon.

that's great, thanks for the quick turnaround

Versioningit v0.3.0 has now been released.


PHOENIX-5252 Add job priority option to UpdateStatisticsTool

LGTM overall, @yanxinyi Can you provide a brief description on the Jira as to how this option works internally so that the users understand exactly how to use it?


Auto numerical axis by data.

Feature Proposal

Datas can't drawed without labels now.
So, i think labels should be generate and scale by datas. Any numerical data should be drawn without a label.
Feature Use Case

https://imgur.com/BxKViLI
I want to build a chart like this without using any labels. The important points here are the LSL and USL values.
Let's assume that x point of the LSL value is equal to 7. 7 does'nt exist in the x-axis on the picture as you can see.
LSL value must be able to drawn on 7 in x-axis of the scaled graph without using any label.
Possible Implementation

In my opinion, there should be a new property for labels. Also, this property can be able to use with any type of chart.

The labels do not need to be specified if the X axis is changed to a linear axis. The scatter chart does this: https://www.chartjs.org/samples/latest/charts/scatter/basic.html
I don't think we want to automatically select the axis type based on the data as that's a major breaking change


'cannot find TOC tags' err when insert new TOC

Hi team,
After I installed MarkdownTOC on sublimetext 3  and then tried to inert new TOC, I got this err:
'cannot find TOC tags'
Traceback (most recent call last):
  File "Y:\Dev\Sublime Text 3\sublime_plugin.py", line 556, in run_
    return self.run(edit)
  File "MarkdownTOC in Y:\Dev\Sublime Text 3\Data\Installed Packages\MarkdownTOC.sublime-package", line 40, in run
TypeError: Can't convert 'bool' object to str implicitly

But I have successfully installed MarkdownTOC on my another PC and it works fine.
Could you pls help with this?

Hi Nick-Alam,
Could you please provide information on operating system and possibly exact version/build of Sublime Text.
Did you install via Package Control or some other way.
jonasbn

Hi @Nick-Alam
I'm so sorry for late reply, but I haven't represent this issue yet.
Please let me know if you still have this problem, and give me more information (how to reproduce / example / etc.).
I'm planning to close this issue in the next week if I won't get any progress.

Close this because of 'no information for reproduce'.


Short solution needed: "Block ip with Lua" (nginx-lua)

Please help us write most modern and shortest code solution for this issue:
Block ip with Lua (technology: nginx-lua)
Fast way
Just write the code solution in the comments.
Prefered way

Create pull request with a new code file inside inbox folder.
Don't forget to use comments to make solution explained.
Link to this issue in comments of pull request.

Block requests by IP address


Stops swallowing of JSON-RPC error details.

Fixes #1.
ethjs-rpc
Thank you for contributing! Please take a moment to review our contributing guidelines
to make the process easy and effective for everyone involved.
Please open an issue before embarking on any significant pull request, especially those that
add a new library or change existing tests, otherwise you risk spending a lot of time working
on something that might not end up being merged into the project.
Before opening a pull request, please ensure:

[ ] You have followed our contributing guidelines
[ ] Pull request has tests (we are going for 100% coverage!)
[ ] Code is well-commented, linted and follows project conventions
[ ] Documentation is updated (if necessary)
[ ] Internal code generators and templates are updated (if necessary)
[ ] Description explains the issue/use-case resolved and auto-closes related issues

Be kind to code reviewers, please try to keep pull requests as small and focused as possible :)
IMPORTANT: By submitting a patch, you agree to allow the project
owners to license your work under the terms of the MIT License.

Coverage remained the same at 100.0% when pulling 1343a7a2667f1d0b96c644f18f2be1e174835229 on MicahZoltu:patch-1 into 640e0a980e3ec5ea3fd8b84daf5bda720fa644ff on ethjs:master.

Will patch now, apologies!

@MicahZoltu my sincerest apologies, this should have been patched a long long time ago.
Should now be available in<EMAIL_ADDRESS>/<EMAIL_ADDRESS>


default/reset value of single reducer is null

default value in reducer must not be undefined. but null is ok.

Coverage remained the same at 100.0% when pulling 058efc1a56b906260df201c3a1b00e25436c5ab5 on fix-reducer-single-default into 04ddc2a85eea96e78985975d4078e3f092cc57c8 on master.


Changing logging level for config transformation and fixing rule

Description

Addressed Array Node replacement
Fixed PipelineDataflowModel transformation from json.
Addressed relative file path for rules and templates.
Changed logging level for config transformation
Fixed rule and template for documentdb

Check List

[ ] New functionality includes testing.
[ ] New functionality has a documentation issue. Please link to it in this PR.

[ ] New functionality has javadoc added

[ ] Commits are signed with a real name per the DCO

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Let's clean up the code as follow up PR. I'm fine with merging existing code to test.


nf3.sendTransaction should check for a live websocket to the Blockchain node

When nf3 is initialised, it creates a ws:// connection to a blockchain node.  This socket can drop over periods of inactivity and that causes nf3.sendTransaction to fail.
Correct behavior: before attempting to send data over the websocket to the Geth node, nf3 should check that it is still alive, and re-establish it if not. Other websockets have not been observed to drop.  This may be because they are more chatty but they should be monitored for similar issues.

Hi @imagobea. I think this is still needed.  At the moment, if a submitTransaction fails then the function will throw.  That's possibly ok, if the calling application handles it, but the calling application would have no way to re-establish a lost connection and therefore this particular error should probably be handled at the nf3 level.

Recreated in nightfall-sdk #55


Elemental encoder streaming

Hello
I open an SRS real-time.conf stream and we trying to stream there via Aws elementary encoder stream, but we all time getting the disconnect by timeout.
Steamer starts publishing sends 5 bloks at the same time, SRS gets 5blocks and waiting 5000 Ms and closing connection by timeout. İs there any options not to break publisher and just to wait  for next blocks ?
According Wireshark on srs server we getting message TCP window 0

Seems network problem.


wld-cli doesn't extract unknown fragment types

Do you have a good way of extracting fragments out of WLDs that are not yet supported?  For example, gequip.wld has 0x26 and 0x27 in it (at least) and the wld-cli extract tool fails to extract them.

Funny you mention it. I'm working on some improved error reporting in the parser right now. The plan, and it seems to be working out so far, is to return a collection of errors containing the fragment headers and unparsed bodies of unknown fragments. And hopefully, if I can convince nom to work with me, any parsing errors encountered in known fragments too!

Commenting out these lines worked as a quick hack to extract a wld file that isn't fully supported. https://github.com/cjab/libeq/blob/477a86524b7b23d06e942d02395f3d3661f3954b/crates/libeq_wld/src/parser/mod.rs#L53-L55


Add Django 1.8 Tests

Story
As a user, I want to be able to use Django 1.8

Well. Clearly, I never got to this.


Solar panels

I believe solar panels need to be added to the blocked materials list at me.badbones69.crazyenchantments.api.events.ArmorListener

Hello, I am not sure what you mean. There are no solar panels in Minecraft.

You must mean Daylight Sensors. I'll make sure to have it fixed in the next update thanks.


Improve pypesa directory structure

Greetings @Kalebu , so this just a suggestion on the package structure improvement. Suggestion list goes as follows;

Use __init__.py to only Import key classes | methods from modules directly into the package namespace. i.e the folder structure could then be;

pypesa/
__init__.py
mpesa.py
mpesa_execptions.py
service_urls.py

__init__.py could also include other important parameters like, version, author, release date, doc-string with a decent sample usage etc etc.
Add a tests folder with the following structure;

tests/
__init__.py
test_pypesa.py

Note, tests/__init__.py can/should be left empty.

Any other structural improvements that i haven't listed/noticed yet.

So to sum up, i think i can working on this if you think it might be a good idea. Also, i think there other code related improvements suggestion(mostly on the OOD), that i will definitely create their own separate issues.

Hi Warren
Thanks for taking  your time putting  all together
I think its great idea,  you can proceed  with the changes

update the documentation  for every  change  you make

Let me  incase of anything
Best regards
Kalebu


Linux: Window doesn't close/quit on first click

On Ubuntu, when attempting to close the Brackets window via the X button, it takes 2 attempts before the window actually closes.

@redmunds Is someone working on this?  It looks like you added 'fix in progress' recently but also cleared the assignee...

The pull request is https://github.com/adobe/brackets-shell/pull/458. This was assigned to me, but I have not been involved with it, so I removed myself.

This bug still exists in Brackets Release 1.2 experimental build 1.2.0-15697 (release 8f82e2a97).
Also, when reopening Brackets, it reverts to a smaller window size even if it was fullscreen before exiting...

Anything blocking the review / merge of the PR? The fix seems to work according to @jasonsanjose

Ubuntu 14.04 - Brackets 1.6 bug still exists.
Does someone have clear tip how to patch it right now?

Still having this issue, Ubuntu 16.04 - Brackets 1.7. I concur with @Mr-Skwiggs - the window does revert to a smaller size after reopening, even if it was fullscreen.

@jextrevor try brackets-electron, https://github.com/zaggino/brackets-electron/releases current version matches latest brackets and works much better on linux (minus live preview functionality)

Looks neat, will wait for some good soul to include it in AUR since I use arch linux...

@axedre you should be able to install AppImage on arch linux as far as I know. More info on http://appimage.org/

Still having this issue on Brackets 1.10 on Ubuntu GNOME 17.04 64bit, I always have to press close two times to exit the program.

Fixed By: https://github.com/adobe/brackets-shell/pull/619


[logstash] feat: add podAffinity settings

Fix https://github.com/elastic/helm-charts/issues/1032

[X] Chart version not bumped (the versions are all bumped and released at the same time)
[X] README.md updated with any new values or changes
[X] Updated template tests in ${CHART}/tests/*.py
[ ] Updated integration tests in ${CHART}/examples/*/test/goss.yaml

Since this is a community submitted pull request, a Jenkins build has not been kicked off automatically. Can an Elastic organization member please verify the contents of this patch and then kick off a build manually?

jenkins test this please

jenkins test this please

Some changes are required to make Black formatter happy in elastic+helm-charts+pull-request+lint-python/835.

Black CI test can be so annoying some time, but it helps keeping consistency in test code format.

👋🏻 @ebuildy Do you think your could rebase your PR and fix the python formatting issues?

ok, I am removing this one and creating a new PR

hey @jmlrt , I have just create https://github.com/elastic/helm-charts/pull/1257 from a fresh branch. Thanks you.


Add JWT default secrets

From: https://github.com/wallarm/jwt-secrets/blob/master/jwt.secrets.list
By @d0znpp / @wallarm
If the license is compatible of course :)

Please open up a pull request

+1


ASSERT_RETURN_ERROR should handle error status even when debug not enabled

Currently, ASSERT_RETURN_ERROR will only check for return success when DEBUG is enabled
See definition here:
\#if !defined(MDEPKG_NDEBUG)
#define ASSERT_RETURN_ERROR(status_parameter)                                  \
    do {                                                                   \
        if (RETURN_ERROR(status_parameter)) {                          \
            DEBUG((DEBUG_ERROR,                                    \
                   "\nASSERT_RETURN_ERROR (status = %p)\n",        \
                   status_parameter));                             \
            _ASSERT(!RETURN_ERROR(status_parameter));              \
        }                                                              \
    } while (FALSE)
#else
#define ASSERT_RETURN_ERROR(status_parameter)
#endif

It ends up causing use cases as such in the codebase, where it looks like we are checking for error, but we are missing it entirely when debug is disabled:
    status = spdm_build_opaque_data_version_selection_data(
        spdm_context, &opaque_psk_exchange_rsp_size, ptr);
    ASSERT_RETURN_ERROR(status);
    ptr += opaque_psk_exchange_rsp_size;

ASSERT_RETURN_ERROR should do one of the following:

Be removed entirely, as ASSERTS should be for configuration mismatches, rather than runtime checks
Actually return the relevant error, and if debug enabled, throw assert before returning error
Keep same behavior, but make it clear that it will only throw assert, and error handling should still be done

I think it would be best to remove (as a part of refactoring ASSERT usage in general). If we are going to keep, I'm a fan of the former, although it should be made obvious that the assert will actually return on failure, since any macro that impacts code flow can cause unintended consequences if impact isn't clear.

ASSERT means to be removed entirely in release build.


Bugfixes

The Go 1.3 failures are obviously expected, I'll turn off the Jenkins 1.3 tests after this is merged.


ERROR: deployment failed: failing invoking action 'provision'

This issue is for a: (mark with an x)
- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

I tried it on both Az CLI inside portal.azure.com as well as in my local system.
both azd up from "Starting from scratch" and "Using existing resources" steps from README throws this error, and deployment fails.

Any log messages given by the failure

I am sharing both types of full stack trace that I had observed:
Log Stack 1:
atanu [ ~ ]$ azd up

Packaging services (azd package)

  |       | Packaging service backendExecuting prepackage hook => /tmp/azd-prepackage-1395958083.sh

up to date, audited 115 packages in 645ms

9 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
  |====== | Packaging service backend
><EMAIL_ADDRESS>build
> tsc && vite build

  |   ====| Packaging service backendvite v4.4.2 building for production...
✓ 1237 modules transformed.
../backend/static/index.html                           0.72 kB │ gzip:   0.38 kB
../backend/static/assets/github-fab00c2d.svg           0.96 kB │ gzip:   0.52 kB
../backend/static/assets/OneShot-daa12333.css          0.83 kB │ gzip:   0.39 kB
../backend/static/assets/index-49ee3353.css            6.55 kB │ gzip:   2.02 kB
../backend/static/assets/NoPage-77feaed1.js            0.29 kB │ gzip:   0.25 kB │ map:     0.41 kB
../backend/static/assets/fluentui-icons-b9d535f4.js    4.32 kB │ gzip:   2.01 kB │ map: 2,767.96 kB
../backend/static/assets/OneShot-2e72308c.js           7.92 kB │ gzip:   2.16 kB │ map:    20.20 kB
../backend/static/assets/index-b2b8058b.js            33.64 kB │ gzip:   7.55 kB │ map:    57.32 kB
../backend/static/assets/fluentui-react-f0f9677f.js  214.82 kB │ gzip:  62.44 kB │ map: 1,005.98 kB
../backend/static/assets/vendor-69a52085.js          751.19 kB │ gzip: 244.79 kB │ map: 3,613.16 kB

(!) Some chunks are larger than 500 kBs after minification. Consider:
- Using dynamic import() to code-split the application
- Use build.rollupOptions.output.manualChunks to improve chunking: https://rollupjs.org/configuration-options/#output-manualchunks
- Adjust chunk size limit for this warning via build.chunkSizeWarningLimit.
✓ built in 14.40s
  (✓) Done: Packaging service backend
  - Package Output: /tmp/azddeploy3219943768.zip

Provisioning Azure resources (azd provision)
Provisioning Azure resources can take some time

  You can view detailed progress in the Azure Portal:
  https://portal.azure.com/#blade/HubsExtension/DeploymentDetailsBlade/overview/id/%2Fsubscriptions%2Ffe12275e-abf8-45a8-9864-abd81109256a%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Fopenai-demo-1691388953

ERROR: deployment failed: failing invoking action 'provision', error deploying infrastructure: deploying to subscription:

Deployment Error Details:
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource 'ff652c54-89fd-5414-b42e-3e3a51cae9d2' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-openai-demo/providers/Microsoft.Authorization/roleAssignments/ff652c54-89fd-5414-b42e-3e3a51cae9d2'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource 'f9b51d98-ac78-5be5-9b2e-f4ce79732aed' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-openai-demo/providers/Microsoft.Authorization/roleAssignments/f9b51d98-ac78-5be5-9b2e-f4ce79732aed'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource '350252ab-5e40-5e2c-927e-2b15a7893bf4' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-openai-demo/providers/Microsoft.Authorization/roleAssignments/350252ab-5e40-5e2c-927e-2b15a7893bf4'.'.
SkuUnavailableInZone: The sku Sku: Standard_ZRS, Kind: StorageV2 is not available in zone.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource '305e1ee8-d95d-54c7-8bbb-1f2a357337fe' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-openai-demo/providers/Microsoft.Authorization/roleAssignments/305e1ee8-d95d-54c7-8bbb-1f2a357337fe'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource 'cc0e9762-7af3-50b9-be07-ba92cbe8de62' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-openai-demo/providers/Microsoft.Authorization/roleAssignments/cc0e9762-7af3-50b9-be07-ba92cbe8de62'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource '78ab4854-54f4-5c73-b5a6-a28f86a9a026' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-openai-demo/providers/Microsoft.Authorization/roleAssignments/78ab4854-54f4-5c73-b5a6-a28f86a9a026'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource '96f96b6f-82ba-58a9-9e77-a0a5861a1ab8' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-openai-demo/providers/Microsoft.Authorization/roleAssignments/96f96b6f-82ba-58a9-9e77-a0a5861a1ab8'.'.
LocationNotAvailableForResourceType: The provided location 'southindia' is not available for resource type 'Microsoft.Search/searchServices'. List of available regions for the resource type is 'germanynorth,qatarcentral,polandcentral,swedencentral,ukwest,koreasouth,canadaeast,jioindiawest,westus3,germanywestcentral,norwayeast,switzerlandwest,switzerlandnorth,westus,westus2,eastus,eastus2,northeurope,westeurope,southeastasia,eastasia,northcentralus,southcentralus,centralus,japanwest,japaneast,koreacentral,australiaeast,australiasoutheast,brazilsouth,centralindia,westcentralus,canadacentral,uksouth,francecentral,southafricanorth,uaenorth'.
LocationNotAvailableForResourceType: The provided location 'southindia' is not available for resource type 'Microsoft.CognitiveServices/accounts'. List of available regions for the resource type is 'global,australiaeast,brazilsouth,westus,westus2,westeurope,northeurope,southeastasia,eastasia,westcentralus,southcentralus,eastus,eastus2,canadacentral,japaneast,centralindia,uksouth,japanwest,koreacentral,francecentral,northcentralus,centralus,southafricanorth,uaenorth,swedencentral,switzerlandnorth,switzerlandwest,germanywestcentral,norwayeast,westus3,jioindiawest,qatarcentral,canadaeast'.

TraceID: 0dbe57fddf8e14fd626f03f6eb202726

Log Stack 2:
azd up

Packaging services (azd package)

  |       | Packaging service backendExecuting prepackage hook => C:\Users\tsa2cob\AppData\Local\Temp\azd-prepackage-1913210062.ps1
  |=======| Packaging service backend
up to date, audited 115 packages in 8s

9 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
  |     ==| Packaging service backend
><EMAIL_ADDRESS>build
> tsc && vite build

  |      =| Packaging service backendvite v4.4.2 building for production...
✓ 1237 modules transformed.
../backend/static/index.html                           0.73 kB │ gzip:   0.38 kB
../backend/static/assets/github-fab00c2d.svg           0.96 kB │ gzip:   0.52 kB
../backend/static/assets/OneShot-0a57a895.css          0.83 kB │ gzip:   0.39 kB
../backend/static/assets/index-a703ca10.css            6.58 kB │ gzip:   2.03 kB
../backend/static/assets/NoPage-d0bd4518.js            0.18 kB │ gzip:   0.18 kB │ map:     0.36 kB
../backend/static/assets/fluentui-icons-df61cf1f.js    4.32 kB │ gzip:   2.01 kB │ map: 2,767.96 kB
../backend/static/assets/OneShot-43cf0699.js           4.89 kB │ gzip:   1.91 kB │ map:    19.25 kB
../backend/static/assets/index-e2fb1672.js            19.08 kB │ gzip:   6.49 kB │ map:    57.32 kB
../backend/static/assets/fluentui-react-d3a943c3.js  212.56 kB │ gzip:  61.51 kB │ map: 1,000.16 kB
../backend/static/assets/vendor-f4aa6f85.js          443.47 kB │ gzip: 151.43 kB │ map: 1,724.15 kB
✓ built in 16.05s
  (✓) Done: Packaging service backend
  - Package Output: C:\Users\tsa2cob\AppData\Local\Temp\azddeploy2552801785.zip

Provisioning Azure resources (azd provision)
Provisioning Azure resources can take some time

? Enter a value for the 'openAiResourceGroupLocation' infrastructure parameter:  1. (Europe) West Europe (westeurope)
? Save the value in the environment for future use No
  You can view detailed progress in the Azure Portal:
  https://portal.azure.com/#view/HubsExtension/DeploymentDetailsBlade/~/overview/id/%2Fsubscriptions%2Ffe12275e-abf8-45a8-9864-abd81109256a%2Fproviders%2FMicrosoft.Resources%2Fdeployments%2Foai-demo-dev-1692369939

  (✓) Done: Resource group: rg-oai-demo-dev
  (✓) Done: Storage account: stv4zk54o7rhsye
  (✓) Done: Search service: gptkb-v4zk54o7rhsye
  (✓) Done: App Service plan: plan-v4zk54o7rhsye
  (✓) Done: Form recognizer: cog-fr-v4zk54o7rhsye

ERROR: deployment failed: failing invoking action 'provision', error deploying infrastructure: deploying to subscription:

Deployment Error Details:
OperationError: Failed to Update the resource. Provisioning state: Failed
ResourceProviderError: An error occurred while sending the request.

TraceID: 5e43686aab8d984127733cb6f955ee3c

Expected/desired behavior

The azure-search-openai-demo should be up and running, but in all my attempts, the deployment fails
with ERROR: deployment failed: failing invoking action 'provision' and different kinds of error stack trace.

first case, I get Auth error, even though I have full contributor access to the subscription
second case, I get deployment failure.

OS and Version?

Windows 10 Enterprise
Azure Cloud Shell --> Bash

azd version?

azd version 1.2.0

Versions

NA

Mention any other details that might be useful

NA

Hmm, that second error is unfortunately quite vague- it looks like it errored when provisioning either App Service or OpenAI. If you click through to the Portal deployment page, does it provide any more details on the error? From that page, you can log a support request, and the support folks can sometimes provide more context. Another thing to try is to run azd up --debug and see if the additional logs help pinpoint the issue.

@pamelafox sure, let me check and get back with more details.

@pamelafox I found one of my colleague at Germany succeeded with the settings:

Select an Azure location to use: as 34. (Europe) West Europe (westeurope)
Enter a value for the 'openAiResourceGroupLocation' infrastructure parameter: as 3. (Europe) France Central (francecentral)

I attempted a fresh new deployment with a fresh new resource group and cloud space, under an active Az Subscription for which I have the 'Contributor' role. On running azd up --debug this time,
still no luck! Here is the error portion from the log stack trace:
ERROR: deployment failed: failing invoking action 'provision', error deploying infrastructure: deploying to subscription:

Deployment Error Details:
InvalidResourceLocation: The resource 'cog-phlwt6piytcgo' already exists in location 'eastus' in resource group 'rg-atanu-dev'. A resource with the same name cannot be created in location 'francecentral'. Please select a new resource name.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource '66c25f40-818e-5f48-bc90-77cccd95aedc' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-atanu-dev/providers/Microsoft.Authorization/roleAssignments/66c25f40-818e-5f48-bc90-77cccd95aedc'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource '33a82b47-dcf3-57fd-855f-70c22db8e027' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-atanu-dev/providers/Microsoft.Authorization/roleAssignments/33a82b47-dcf3-57fd-855f-70c22db8e027'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource 'bd99a5e2-4185-5dc9-8ad2-78e293a1fca9' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-atanu-dev/providers/Microsoft.Authorization/roleAssignments/bd99a5e2-4185-5dc9-8ad2-78e293a1fca9'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource '1b972820-439a-59e4-91d9-1debc112e11e' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-atanu-dev/providers/Microsoft.Authorization/roleAssignments/1b972820-439a-59e4-91d9-1debc112e11e'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource 'f4113f5e-a644-5d4c-988c-78bc576a4065' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-atanu-dev/providers/Microsoft.Authorization/roleAssignments/f4113f5e-a644-5d4c-988c-78bc576a4065'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource 'd5980b96-aec6-595f-83a0-6a61d619b05e' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-atanu-dev/providers/Microsoft.Authorization/roleAssignments/d5980b96-aec6-595f-83a0-6a61d619b05e'.'.
InvalidTemplateDeployment: The template deployment failed with error: 'Authorization failed for template resource '24ab7668-b23a-5b14-9b3e-51b31cb70a5c' of type 'Microsoft.Authorization/roleAssignments'. The client<EMAIL_ADDRESS>with object id '8aa35960-f0ab-417d-95b8-31bc85ccf0d7' does not have permission to perform action 'Microsoft.Authorization/roleAssignments/write' at scope '/subscriptions/fe12275e-abf8-45a8-9864-abd81109256a/resourceGroups/rg-atanu-dev/providers/Microsoft.Authorization/roleAssignments/24ab7668-b23a-5b14-9b3e-51b31cb70a5c'.'.

TraceID:<PHONE_NUMBER>0000000000000000000000
2023/09/14 05:02:55 cloudshell.go:14: running in Cloud Shell

Could you help me out on why I keep getting "Authorization failed" or the above error, even though I have access?

Per the readme, you need more than Contributor level access. Can you be given RBAC access to that subscription or even to just that resource group?

Hey @pamelafox
Thanks a lot for all the support.
After getting owner access, and deploying az resources to the above-mentioned regions, the demo works for me inside the corporate framework!
Just one more thing,
Is it possible to work with data sources other than PDF?
I have mentioned the use cases in this issue: #633
I believe #478 and #633 would be really helpful use cases for many, and looking forward to these features.
In the meantime, if you can help me with a way around for #633 it would be of great help.
Thanks!

Great, glad you were able to deploy! I'll close this issue as I've replied on the other one.


Added the ability to configure jersey properties via configuration

You can now specify something like this in the config.yaml:
    jerseyProperties:
      com.sun.jersey.spi.container.ContainerRequestFilters: com.hmsonline.sui.webservice.logging.LoggingFilter
      com.sun.jersey.spi.container.ContainerResponseFilters: com.hmsonline.sui.webservice.logging.LoggingFilter

This is a simple key/value map of properties set on the environment jersey properties.
This new property is not required.

LGTM

LGTM -- I'll cut a release of this later today as well as a PR for Dropwizard 0.7.

LGTM, merging.


Expandable: Fade out section when 'closed'

Issue
The abrupt cut-off of the expandable section bugs me (looks like a rendering glitch), especially when it cuts off in the middle of a line.
Change
In addition to the existing 'More' button, we fade out the section to provide additional visual cues.
Approach
This solution doesn't require the background color to be known, so it will work regardless where <Expandable> is used, or whatever color-scheme is chosen.
However, it does utilize non-standard css -- for older browsers, it should simply cut-off like the before.

Test

[x] Firefox (working)
[x] Chrome (working)
[x] Opera (working)
[x] Edge (fails with style and grace)
[ ] Internet Explorer (couldn't get lbry to load at all)

@infinite-persistence this is great!


Creating custom keyboard

Hello, any tutorial for creating custom keyboard? for example i use keyboard type "number" and "phone" but i want to make like calculator keyboard + just 0-9 keyboard and etc..

@Nurgunkalol thank you for your interest in NativeScript.
Take a look at this blog post that shows how to solve this issue on iOS.
the repository is for logging issues, bugs and features related to nativescript-angular plugin.
For how-to related questions please use the community channels like the NativeScript forum, SO or NativeScript community slack


Add network param to voting strategy form

First step would be to just add Network select when setup a voting strategy, then make hub works to store and query this information

Next Step - Score API / Snapshot.js use the network property in the strategy to calculate the voting power using blockfinder.


Application level logging

Currently, the frontend app is configured to log information on the server level (node). This makes debugging difficult.
How would you go about implementing application-level logging? There is logger already declared  react-redux-boilerplate/server/util/logger.js. Should I add another lib (perhaps winston) or rework logger.js ?
@flexdinesh Could you share your thoughts about that?

@piotr-roslaniec I'm not really sure if I understand what you mean or how the logging could be enhanced for the frontend. But the logger declared in react-redux-boilerplate/server/util/logger.js is not really of much use. It's just used to print on the console whether the server has started and on what port it's running on. In one of the other projects I am working on I have replaced the logger with winston already and it's really good for server side logging.
Perhaps could you clarify a bit more on how we could implement application level logging for frontend and how these logs could be used for debugging?

@flexdinesh
Thank you for your response.
Sorry for being so indirect earlier, but I hadn't thought it through
I had some issues with debugging my project in production lately. I thought I'd go about implementing some logging solution like PaperTrail or Sentry to detect client-side errors (not sure if I used nomenclature correctly). I thought I'd ask around for some recommendation and maybe PR this later if someone finds it useful.

@piotr-roslaniec Implementing a client side logging with PaperTrail and Sentry sounds like a great idea for actual projects. But for boilerplates, the scope is too wide and not everyone would want a client side logging tool and we want to keep this boilerplate to as minimal as possible. Also, these tools would involve an account setup and API key to use it with the application, which is again a strenuous process to quickly start with the boilerplate. Perhaps if we could create an example optional code and is turned off by default in such a way that people could use that logic only if they decide to use it. That makes more sense in the context of a boilerplate. I'd very much welcome a PR if we could keep this feature optional and have a solid example to state its purpose.

Closing due to inactivity


[Todo List] shortcut to add item

Extension
https://www.raycast.com/maggie/todo-list
Description
It would be nice to expose the ability to add an item to a todo list by shortcut. Could be used in combination with clipboard or more complex integrations (personally, I want to be able to write a script to check my Obsidian notes once a day and pull any lines with #todo into my Raycast todo list).
Who will benefit from this feature?
everyone who needs todos
Anything else?
No response

@ancepsinfans this is a great idea, would be nice to have such feature.


Creation of new Objects (Language & Contact Language) & fields to support Languages in HEDA

Note: FOR INTERNAL USE ONLY. For support questions, please use the HEDA Chatter group in The Power of Us Hub.
For feature requests, please use the Ideas tab in The Power of Us Hub.

Do you want to request a feature or report a bug?

Feature

What is the current behavior?

The current functionality does not exist in HEDA.

What is the expected behavior?

This issue will result in new objects & fields and not any new logic/functionality. The user will interact with these objects the same way they interact with other objects in Salesforce.
A user can navigate to a Contact record to view or set their preferred language. There may be other languages that the Contact can speak that will also be tied to that Contact, as well as their proficiency in those languages.

What is the motivation / use case for changing the behavior?

These new objects & fields will allow an organization to track what languages are spoken by students/staff/parents, their fluency with each, and where they are spoken.
The language record will track the preferred (primary) language of a Contact in HEDA. The ability for an educational institution to know that Student John's primary language is English, but the best way to communicate with his parents' is Spanish. The ability to know the fluency of an instructor's languages in a global institution being taught in multiple countries or countries with more than one language; e.g. Belgium with Flemmish and French.

**lurch: add

Tracking W-031371

Included in beta release 1.72 (Beta 6)

Included in production release 1.72