{% extends "page_with_title_and_lede.html" %} {% load i18n %} {% block head_title %}Staying in Town for North Bay Python{% endblock %} {% block heading %}Staying in Town for North Bay Python{% endblock %} {% block body_class %}attend{% endblock %} {% block lede %} You'll enjoy North Bay Python more if you stay in town! Staying in a hotel within walking distance from the venue, and near dozens of great restaurants, makes it much easier to meet other attendees and get that much more sleep every morning. {% endblock %} {% block content %}
How you get here depends, naturally, on where you're coming from. Broadly, your options are: flying, cab rides, public transit, ride sharing apps, or renting a car.
Public transit to Petaluma is currently not great. You can take the 101 bus operated by Golden Gate Transit from downtown San Francisco, or south from Santa Rosa. Depending on sponsorship, we hope to run a free shuttle with BART and Caltrain connections for people from further out of town.
SMART, the new train service that runs along the 101 corridor, recently started operations. SMART is not suitable for getting to North Bay Python if you travel on weekends, as the first train leaves after proceedings start. SMART may be a better option than taking the bus between San Rafael and Petaluma if you travel up on weekdays.
If you're driving up, downtown Petaluma is at exit 472A on Highway 101, 35 miles north of the Golden Gate Bridge.
All parking is free in Petaluma, however near the Mystic, street-level parking is time-limited. All-day parking is available at street level west of 5th St (towards 6th St), and at the the undercover garages at Theatre Square and at Keller St. Both garages are in short walking distance of the Mystic.
You'll probably want to stay at one of the hotels in Petaluma, but there are options nearby in Rohnert Park, or in Sonoma Valley if you'd like an extra dose of scenery. If you're feeling adventurous, you could even stay out on the coast in Bodega Bay or up in Santa Rosa. But staying in Petaluma near the venue means you'll have a very walkable weekend; you won't need to worry about driving or getting rides at all!
Check out our Where to Stay page for more details on hotels and other options, including links to discounted rooms.
All the official events will take place on Friday, Saturday, and Sunday, from November 1st to 3rd, so we recommend you arrive Friday afternoon (at the latest) and depart on Sunday evening (at the earliest).
Many people choose to arrive Thursday evening and depart Monday morning, for convenience. Lots of attendees choose to start their visit even earlier, or stay longer, so they have time to explore our landmarks, expansive parkland, and world class hospitality.
Petaluma is right at the intersection of Sonoma, Napa, and Marin Counties. This three county region, which makes up the North Bay, is famous for its parkland, forests, agriculture, fine food, wine, beer, and spirits. There's a lot to do here and we encourage you to explore while you're here!
We've prepared a food guide which will help you find good eats in Petaluma, but if you're sticking around beyond the conference, you'll want to check out your favorite food or restaurant guide. We've also put together a transit guide to help you get around town once you're here.
From family-friendly hikes to museums, wine tours to wind swept beaches, and lots of kid-friendly attractions, we encourage folks to bring their families along and stay as long as they can! We will offer free childcare for kids from 6 months to 12 years of age during the conference.
If your employer isn't paying for you to attend North Bay Python, then you should apply for financial aid. If you're hesitating about applying for financial aid, don't.
We're a 501(c)(3) nonprofit and our priority is accessibility. Our goal is for attending North Bay Python to cost about as much as staying home. Learn more about our financial aid program, including application process, timelines, and policies. Don't worry, it's easier than it sounds!
{% endblock %}


About
This website is to aid students in selecting their class schedules for the upcoming semester. This website generates schedules for students based on the courses the user selects and each course's availability.
Technologies Utilitized
Node.Js
JavaScript
HTML5
CSS3
Contributors
Kyle Ladd Developer
Morrisville State College Web Development '14 Alum.


React-Grid-Layout Demo 2 - No Dragging
View project on GitHub
View this example's source
⇠
View the previous example: "Basic"
⇢
View the next example: "Messy"
This particular example has dragging and resizing turned off.


Boots Black Friday deals 21 Nov 2019, 16:01 by Ian Mangan Paco Rabanne 1 Million For Men Paco Rabanne 1 Million For Women Panasonic Nanoe Hair Dryer Ted Baker gift set Jack Wills Gym Bag Gift Set


Welcome To Empathy Learning Socially-aware Agents (E.L.S.A.)
Please Select Your Chat Experience
{% for chatbot in chatbots.values() %}
{{ chatbot.name }}
{% endfor %}
Chat History
Please Rate Your Chat Experience!
Thank You!
We appreciate your feedback!
Here's Your Mechanical Turk Code:
Please copy it and paste it into Amazon Mechanical Turk
Are You Sure?
We recommend sending at least three messages.
Welcome to E.L.S.A!
You're about to chat with an exciting chatbot! Before you begin, please read these quick instructions.
It Takes Time to Get to Know You!
In order to give the chatbot a chance to get to know you, please write at least THREE lines before ending your session.
Rate As You Go!
During chat, you can rate responses using "upvote/downvote" buttons next to each line the chatbot sends. Please use them to let the bot know how it's doing!
Chatbots Don't Know Every Word
During the chat, if you say a rare word or make a spelling mistake, the bot will see your word as "<unknown>".
It may also use "<unknown>" when talking about words outside its vocabulary.
Warning: May Contain Adult Content
These bots are trained on movie dialogs and Reddit data, and sometimes say inappropriate things. Chatter discretion is advised.
Let Us Know How The Chat Was
When you are done chatting, please rate your chat experience by pressing the button in the top right of the screen.
{% if study.is_mturk %}
After You Rate, Keep Chatting!
Once you have replied to the bot at least THREE times and feel you're done with your conversation, you can feel free to rate the current bot and go on to the next chat bot.
Please rate as many bots as you can so we can learn which ones work best! We really value your opinions and feedback!
Don't Forget Your Codes!
After you are done talking to EACH chat bot, we will generate and display a code. Copy that code and enter it in the Amazon Mechanical Turk HIT interface.
You will receive one code per bot, remember to copy them all!
{% endif %}
You're All Set!
Thank you for participating!
Welcome to Empathy Learning Socially-aware Agents (E.L.S.A)
You're about to chat with an AI chat bot!
But first, we want you to know we value your privacy and data. So please read and agree to the following consent form before participating.
Informed Consent
[[ EDITME You should put your informed consent content here ]]
Please confirm you agree to these conditions and that you are between 18 - 64 years of age.
{% else %}
No Chatbots Found!
Sorry, it looks like we disabled the chatbots you're looking for.


PyParsers Reference¶
This module contains the core of the pyparsers micro framework to develop DSLs with python (this is actually the only file you’ll need to get started:its truly micro).
 Therefore, it also strives to be minimalist and provide a few powerful abstractions to let you get started without having to go through the burden of learning yet an other parsers engine.
Note
Conceptually, this library was inspired by the Scala parsers library.
Author: X. Gillard
-
class
pyparsers.
Failure
(pos, reason)[source]¶ A parse result encapsulating the case where the stream of token couldn’t be parsed
-
class
pyparsers.
ParseResult
(pos)[source]¶ This is the abstract class that describes the kind of objects to be returned by the parser apis.
-
position
()[source]¶ Returns: the position in the stream of tokens up to which this result accounts for.
-
-
class
pyparsers.
Parser
(fn, action=<function identity>)[source]¶ An actual parser object that can be combined with some other parser to build more advanced grammars
-
class
pyparsers.
Success
(pos, value)[source]¶ A parse result encapsulating the case where a subsequence of tokens has correctly been parsed
-
class
pyparsers.
TokenStream
(tokenizer, text)[source]¶ A stream of token implemented as a thin wrapper around a list of element.
Note
Although you might want to implement an other (ie. more memory efficient) TokenStream, you should make sure an implementation of the __hash__ method as well as some of the methods from the container protocol are available. This way, you should be able to make sure all the parsers defined in the library (or the ones you define) are interoperatable with your new token stream implementation.
-
class
pyparsers.
Tokenizer
(whitespace='\s', punctuation='$^')[source]¶ A configurable tokenizer that lets you decide what should be considered whitespace (whitespaces are dropped) and what should be considered punctuation (kept).
-
class
pyparsers.
VariadicActionParser
(rule, action)[source]¶ A parser implementation that accepts variadic actions. That is to say, a parser producing a list or tuple that will be broken down to several parts before being passed to the action function.
The typical use for this class lies in the implementation of the sequence: as every other parser builder, it returns a parser whose action can be parameterized but it makes sure that the given action receives a series of arguments rather than a list.
-
pyparsers.
leftrec
(fn)[source]¶ Decorator that activates the packrat left recursion support for this function. .. Note:
This implementation is based on the paper by Warth, Douglass and Millstein
-
pyparsers.
list_of
(rule, sep=', ', action=<function identity>)[source]¶ Generates a parser for the case when you want to parse a list of items separated by a token (or rule) of your choice.
Note
The parse list has necessarily the following structure: rule sep rule sep rule sep ... rule
Note
Given that it tries to recognise a list of items separated with some given separator, an empty list will be rejected with a Failure.
Although many other implementations are possible, this feature was included simply because parsing lists of items is a very common use case. Therefore it was considered useful to free developers from developing their own list parsing utility function.
Parameters: - rule – the rule parsing items in the list
- sep – the rule describing the separation between items
- action – the action to be applied on the parsed list of elements
-
pyparsers.
memoize
(fn)[source]¶ Decorator that memoizes the results of the function calls. This is pretty useful if you intend to implement a packrat parser.
-
pyparsers.
one_of
(*enumerated, *, action=<function identity>)[source]¶ Generates a parser that recognizes one of the specified given values :param enumerated: the list of the values recognized by this parser. :param action: the action applied to the parsed token
-
pyparsers.
optional
(rule, action=<function identity>)[source]¶ Generates a parser that possibly recognizes an optional string
Parameters: - rule – the rule to be possibly matched
- action – the action to be applied on the result
Returns: a parser that recognises rule 0 or one time.
-
pyparsers.
parse_all
(text, axiom, tokenizer=<pyparsers.Tokenizer object>)[source]¶ Utility function to parse all the text of the given input text :param text: the text to be parsed :param axiom: the axiom (base rule of the derivation) that generates the root of the ast :param tokenizer: an optional tokenizer customized to your needs :return: the value corresponding to a successful parse of the text :raises: an exception if not all text could be parsed
-
pyparsers.
parser
(fn)[source]¶ Wraps a function to make it a parser (useful to implement recursive call)
-
pyparsers.
regex
(pattern, action=<function identity>)[source]¶ Generates a parser that recognizes the given regex. :param pattern: pattern recognized by this parser. :param action: the action applied to the parsed result
-
pyparsers.
repeat
(repeated, min_occurs=0, max_occurs=inf, action=<function identity>)[source]¶ Generates a parser for a rule that can be repeated.
Parameters: - repeated – the rule that can happen many times
- action – the function that receives the list of inputs as param and treats them all.
Returns: a parse result containing the processed output of the many results of repeated
-
pyparsers.
sequence
(*parsers, *, action=<function <lambda>>)[source]¶ Generates a composite parser for a sequence of tokens.
Parameters: - parsers – a sequence of parsers that must be recognized
- action – an action to be applied to the sequence of outputs recognized by the parsers. (note: one param per recognized value)
Returns: a parse result containing action( *( *parsers.value() ) ). (In case the result is a failure, action is obviously not applied)


Press on the braille dots to make them dark thereby selecting the raised dots. The light dots simulate dots that are not raised whereas the raised ones are shown in the dark colour. By pressing the convert button below, the inputted braille is translated to the English character it represents. There exist thirty-eight combinations of raised dots that are not English characters. Can you find them all?
Select the raised dots by making them dark and representing the correct braille image. If the selected braille does not match the correct letter, the correct letter will be shown.
Enter text below for it to be translated into braille. Spaces will be translated as blank spaces, however, other characters that are not English letters cannot be displayed.


Visit the official government website for more information Click Here
‘Seva Bhoj Yojna’ is a Central Sector Scheme of the Ministry of Culture, Government of India. It envisages to reimburse the Central Government share of Central Goods and Services Tax (CGST) and Integrated Goods and Service Tax (IGST) so as to lessen the financial burden of such Charitable Religious Institutions who provide Food/Prasad/Langar (Community Kitchen)/Bhandara free of cost without any discrimination to Public/Devotees.The scheme is being implemented from 01.08.2018 with a total outlay of Rs. 325.00 Crores for Financial Years 2018-19 and 2019-20. Free ‘prasad’ or free food or free ‘langar’ / ‘bhandara’ (community kitchen) offered by charitable religious institutions like Gurudwara, Temples, Dharmik Ashram, Mosques, Dargah, Church, Mutt, Monasteries etc. Financial Assistance will be provided on First-cum-First Serve basis of registration linked to fund available for the purpose in a Financial Year.


Tests whether playback rate control is supported.
If you can independently adjust the MO playback rate, without warping the pitch, then the Reading System supports rate control.
Tests whether playback across spine items is supported.
When this chapter has finished, playback should automatically continue onto the next chapter.


play
pause
mute
unmute
max volume
Status:
Update Required
To play the media you will need to either update your browser to a recent version or update your
Flash plugin
.


Hello
Welcome to Markdown Notebook! Markdown Notebook is a blogging solution that renders Markdown documents client-side, in Javascript. Unfortunately, you don't appear to have Javascript enabled, and Markdown Notebook doesn't actually have any server-side code, which makes things a bit difficult.
If you don't want to turn on Javascript, take note of the filename after the "#" in your URL bar above, and find that file in this Github repository, where you will be able to read it as formatted by Github. You can also browse that repository for interesting posts, or just clone it and look at it in your terminal if you prefer your Markdown unrendered.


Putting people first and involving them in the design process to arrive at solutions with a pre-defined product-market fit.
Bringing design to people and organisations, from business cards to new brand identities and strategies.
Many "experts" claim to know innovation and design, but are they certified? Innovation and Design is not just my degree.
It is my forte passed down from real industry leaders.
I deliver educational content to people all over the world, sharing my learnings and experiences to help others succeed.
Professionally enhance your business appearance with design
Are you launching into a new era? Let's take your branding to the next level
Starting up or scaling? Let me help you get to the next stage of your business
Your personal innovation and design advisor for long term business growth
NOTE Prices are indicative and represent minimum costs, depending on your requirements these prices may or may not increase. Certain projects requiring extra resources will incur additional fees for materials, labour or both. As I strive to bring my best attention and care to each project I take on, there may be instances where your project request will be delayed or rejected either due to holiday periods or increased demand for my services. Please note that should such situations arise, I will respond to you as quickly as possible and work towards a win-win outcome.
Project timeframes can be discussed with me privately to meet your requirements, depending on the scale and complexity of the project there may be increased fees upon completion.
Steven Wang currently serves as Business Designer at How Migh We, and is a member of: the Design Futures Initiative, OpenIDEO, and Young Aussie Entrepreneurs.
Previously, Steven was the Sponsor for the 2018 IUFV Asgardia Education Conference and had been involved with the organisational development and innovation advocacy campaigns of Asgardia for over 2 years.


I've been looking all over the web for information on "Non-Intrusive Load Monitoring", which seems to be an emerging hot trend in the commercial/academic sector.
Basically, it's a device that sits on a home wall socket (or in the electrical panel), and montiors voltage/current fluctuations. It's smart enough to distinguish between fluctions caused by a coffee pot (for example) and a TV set being turned on. By using a learning algorithm and signal processing techniques, it's able to discern when these devices are turned on/off, and how much power's used by each device.
There's several initiatives that have pursued this, including:
Electrisense: http://ubicomplab.cs.washington.edu/wiki/ElectriSense
These guys created a company called "Zensi", which was promply bought by Belkin:
http://www.belkin.com/pressroom/releases/uploads/04_21_10ZensiAcquisition.html
There's a whole number of other academic pursuits and companies that have recieved DOE grants, etc. if you search google for "Non-Intrusive Load Monitoring".
The closest commercial offiering I could find with an actual product is Navetas Home Energy Montior out of the UK: http://www.navetas.com/products/showcase/energy-monitor/
Even that isn't for sale, yet. The Belkin product seems to still be in stealth mode, but it's probably going to hit stores in the next few years.
----------------
Which leads me to the question - is there anything in the home-hobbiest arena that's using this kind of technology? The hardware these things are using is not at all new, pretty much what's documented on this site.
What's unique is the algorithms or "software brains" behind how the signals are processed to identify appliances.
The beauty of Open Source however is the ability to crowdsource solutions, and this problem is fortunately a software one. Software solutions are often easier to create open solutions for, as tweaks only require more code which is cheaper to implement than open hardware (which already exists).
Is there anything in the home-hobbiest arena that's using disaggregate, signal processing Non Intrusive Load Monitoring algorithms?
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
The NILM project started in the 80's at the MIT by George W. Hart and his colleagues (he called it NALM though "Nonintrusive Appliance Load Monitoring). The first approach, that was patented by them, used hardware to estimate any power step change and managed to locate mainly resistive loads with a clear step change. The approach in Washington is to monitor the noise and harmonics that resonates over the phase when appliances are turned on or off or change their state. This approach requires high frequency monitoring and there might be a good ADC to translate this for the Arduino. A combination of these two approaches is currently being persued at MIT to detect and monitor large loads like climate controlling units in commercial buildings.
I have written a small moving window analysis of the incoming data for my arduino. The step changes are followed and once they are above a certain threshold they are matched against known measurements. I haven't had time to tidy up the code but I think you'll be able to follow it. Right now it only tracks one entity of power and only one phase but you can easily add the other measurements to get a more sophisticated recognition. If you have any questions or suggestions please let me know.
PS. Finding appliances today cheaply (without high frequency measurements) is really hard today since many of our appliances consume power very differently depending on how they are used and therefore blend together in measurements. You should be able to find the bigger appliances though with this code (washing maschine heater, toaster, electric kettle... - which is still fun )
cheers!
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Schepparn, that's some pretty cool implementation!
Approximatation of the larger loads is usually "good enough" to track and trend the majority of energy loss. Of course, it won't capture every vampire load but will catch the big players. And even if it's got to be "hand tuned" to certain appliances over a week or two (something big corps. would NEVER sell) it's still something to explore and learn about the process.
Now I gotta buy the hardware! I've been exploring this field for a few months and the only option I resigned to was an army of Tweet-a-watts. Maybe this combined with a few tweet-a-watts will take care of the vast majority of loads. Now to find the time...
Thanks!
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
I'm interested in load disaggregation so as to be able to infer activities of daily life of the elderly living alone.
I have been running a small scale trial to collect household power and water use data. I have studied disaggregation schemes and would like to co-operate with like minded individuals to develop effective open source algorithms.
I've been using current only monitoring because I've been using an off the shelf monitor from Current Cost. I believe that current and voltage monitoring, probably with 24 bit resolution and maybe transient acquisition may be required to effectively disaggregate all but the major appliances. One of the problems is that I want to have a sensor that users can self install without having to employ an electrician so getting voltage date is hard. Onzo and Navetas do this (I think using different approaches) but it all seems to be tied up in patents. Onzo have also done lots of work on disaggregation in the cloud. I'd prefer to see this done in the home - and the system should, ideally, learn appliance characteristics from acquired data rather than from an appliance database and user configuration.
Any ideas welcome.
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Just a very quick reply for now (would love to write more but am racing to meet a deadline!)
I'm 1 year into a PhD on disaggergation and my ultimate aim is to produce an open-source software package for disaggregating aggregate data from off-the-shelf meters like Current Costs and emonTXs (and "proper" smart meters when they start to be installed). I'd love to help out (or collaborate!) in any way... if you're based near London then perhaps we could discuss over a coffee?
I agree that being able to resolve changes of 1 or 2 watts (or less) is probably important. As is having readings of both real and reactive power.
I wrote a series of three blog posts on disaggregation here: http://jack-kelly.com/taxonomy/term/104
And here's my literature database: https://www.zotero.org/jack_kelly/items
Doing disaggregation well isn't easy (and no body is able to do it with anything like 100% accuracy for all appliance types yet)
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
I think some appliances will have a definite signature (a lamp, for example). Others will have more than one signature (a dehumidifier will probably have standby, running, and defrost modes, but all will be a predictable load. But then you come to devices like laptops where playing a game or watching a full screen movie (no, not THAT type!) can cause the standard power consumption to double or more.
So I think a mixed approach would be needed. Central monitoring will be fine for certain loads, but you're going to need appliance based monitoring for devices like computers, TVs, etc.
Having said that, a signature isn't just current - it's also phase angle, and that might help to determine what a particular load is.
 If you're content with knowing a bulb has been switched on somewhere then it's easy, but if you want to know which particular bulb it is then the monitoring is far more complex.
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
If you'd like to see signatures of lots of different appliances then see http://www.tracebase.org/
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Been a year or so - glad to see there's activity on this front!
Jack - awesome work! Best of luck with your PhD process and I sincerely appreciate your open sourcing this work. It takes a lot of time to develop the algorithms involved and having even a basic open source implementation of NILM would really help the effort along!
Have you been in touch with Mario Berges or Zico Kolter at Carnegie Mellon University in the US?They are both working on the NILM problem indepth and have begun to address the issue of the lack of data sets.
Mario Berges: http://tbc.ce.cmu.edu/people/faculty/berges.html
Zico Kolter: http://www.cs.cmu.edu/~zkolter/
1st International NILM Confernece abstracts/slides: http://www.ices.cmu.edu/psii/nilm/agenda.html
Tons of great resources there.
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Hi Electron,
Yes, as you say, Zico Kolter and Mario Berges are both doing amazing work. I'll been following their research output for a while now. All great stuff!
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
By the way, if you are based in the UK and are working on some aspect of smart meter disaggregation / Non-Intrustive Load monitoring then please get in touch; we're hoping to plan a UK disaggregation meetup ;)
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
It looks like the Belkin stuff mentioned in the original post is getting closer to being a product (for both power and water):
http://www.belkinbusiness.com/everything-connected
http://www.belkinbusiness.com/echo-water-0
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Many thanks for those interesting links!
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Just a very quick note that a few of us are toying with the idea of setting up a wiki and/or online community to support electricity disaggregation (aka NILM / NALM / NIALM) researchers. Some more details here.

We have had Solar Thermal panels added, and are having problems with the Wagner control system. Our installer has done his best, but has not fixed it yet. Thus I am attempting to monitor and log the temperature profile of our heat store cylinder that supplies heat for DHW and the underfloor heating. That looks pretty straight forward with a Raspberry Pi and I am well on the way with that bit.
It would be nice to add into the log the state of the underfloor heating pump (on/off) and the solar thermal circulation pump (on/off). In searching this site I have only come up with this Non-Intrusive Load Monitoring thread. It has been very interesting (if mostly beyond my level of knowledge) but i s there a thread that looks at what I am seeking? Alternatively, can any of you experts point me to a nice road tested and proven way of generating a signal for the Rapberry Pi?
Many thanks.
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
I would suggest connecting a mains coil relay in parallel with each pump and using the isolated contacts to connect to your processor to log the on and off state of each pump.
Simple , safe and relatively low cost. suitable relays on ebay and the usual suspects.
HTH
Bob
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
We've just started work on an open source software toolkit for doing disaggregation: nilmtk. It will be a good few months before it does anything useful though ;)
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Wow, the screenshot you have there on your github https://github.com/nilmtk/nilmtk/ of load disaggregation looks amazing, is that being generated now?
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Hiya! Ooops, no, we should definitely make it clearer that nilmtk can't do that yet! It was just an illustration of what NILM can do ;) Sorry, I'll make sure we get that clearer!!!
Some time soon (2-3 months? Maybe 6 months) nilmtk should be able to do that, though ;)
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Hi,
I am Nipun and I am one of the contributors of nilmtk https://github.com/nilmtk/nilmtk/
The image used comes from my prior publication:
Nipun Batra, Haimonti Dutta, Amarjeet Singh, “INDiC: Improved Non-Intrusive load monitoring using load Division and Calibration”, to appear at the 12th International Conference on Machine Learning and Applications (ICMLA’13) will be held in Miami, Florida, USA, December 4 – December 7, 2013 [Preprint] [IPython notebook]
The entire code is made available as open source and you may refer to the IPython notebook linked above.
Thanks for interest!
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
I would like to bump this up in sight again. I was searching for ways to determine which devices are on a single ct and separate them from each other. After reading some papers and interesting webpages, I came across the nilmtk framework. I think it has very high potential.
I am thinking of building an exporter/importer for emoncms to nilmtk. I already did some basic tests with the export to csv function and importing that data into pandas, but I will need to make some specific conversions in order for the framework to work with the dataset. If anybody else is interested in joining the research/testing, I might be uploading it to github.
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
I am also very interested in trying this...
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
Definitely looking into this when I have some time.
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
I am interested too.
I am actually testing for work purpose a commercial nilm device that I cannot name and would love to have OS win...
Re: Non-Intrusive Load Monitoring - one sensor, appliance level detail
I've stumbled across this before but have never looked deeper than their webpage: http://www.smappee.com/uk/


Real World
`Prev `__
R
`Next `__
--------------
**Real World**: n.
1. Those institutions at which ‘programming’ may be used in the same
sentence as ‘FORTRAN’, ‘\ `*COBOL* <../C/COBOL.html>`__\ ’, ‘RPG’,
‘\ `*IBM* <../I/IBM.html>`__\ ’, ‘DBASE’, etc. Places where programs do
such commercially necessary but intellectually uninspiring things as
generating payroll checks and invoices.
2. The location of non-programmers and activities not related to
programming.
3. A bizarre dimension in which the standard dress is shirt and tie and
in which a person's working hours are defined as 9 to 5 (see `*code
grinder* <../C/code-grinder.html>`__).
4. Anywhere outside a university. “Poor fellow, he's left MIT and gone
into the Real World.” Used pejoratively by those not in residence there.
In conversation, talking of someone who has entered the Real World is
not unlike speaking of a deceased person. It is also noteworthy that on
the campus of Cambridge University in England, there is a gaily-painted
lamp-post which bears the label ‘REALITY CHECKPOINT’. It marks the
boundary between university and the Real World; check your notions of
reality before passing. This joke is funnier because the Cambridge
‘campus’ is actually coextensive with the center of Cambridge town. See
also `*fear and loathing* <../F/fear-and-loathing.html>`__,
`*mundane* <../M/mundane.html>`__, and
`*uninteresting* <../U/uninteresting.html>`__.
|image0|
()
--------------
+------------------------------+----------------------------+----------------------------------+
| `Prev `__ | `Up <../R.html>`__ | `Next `__ |
+------------------------------+----------------------------+----------------------------------+
| real user | `Home <../index.html>`__ | reality check |
+------------------------------+----------------------------+----------------------------------+
.. |image0| image:: ../graphics/cobol.png


We enhance content!
- facilitate compelling deliverables
- morph intuitive bandwidth
- benchmark intuitive functionalities
- incubate visionary convergence
- reinvent proactive e-tailers
- whiteboard strategic synergies
- expedite open-source partnerships
- monetize mission-critical networks
- empower front-end users


Place of Work: Dell Panama - Arraijan EUS
Role: Client Technical Support Associate for Anthem formely known as Wellpoint
Time Worked: December 2012 - March 2013
Key Functions:
- Password resets, account activation through Active Directory.
- Basic tech support.
- Remote desktop support, and general help desk tasks
- Registering and following up on service requests, and channeling issues to their correct destination, for elevation or further follow up.


ES6 Blog Posts
1 posts, page 1 of 1
-
Building Apps with Riot, ES6 and Webpack
Feb 10, 2015I arrived at Riot after reading the Muut Frameworkless JavaScript blog post – read this first! The Muut programmers have put their money where their mouths are and built Riot, a minimalist Reactjs-like library called Riot for building reactive UI components.
Reading the Riot documentation the thing that struck me was how easy Riot is to understand – there are very few new terms and concepts to learn compared to React (which, to be fair, is also straight-forward compared to e.g. Polymer and Angular).
To help me learn Riot I ported my React Flux Backbone Todos Example to a Riot Todo app. This post is a compilation of notes I made documenting the experience.


faisal@User
Homepage
Hand Made
Computers
Cell Phones
Tools
Item Name
Item Name
Item Description
Added Date
:2016-10-10
price
:70
Made In
:USA
Category
:Category
Added By
:Username
Login
or
Signup
To Add Comment
Copyright 2017 EliteCorp ©


Yeo and Burge 2004, http://www.ncbi.nlm.nih.gov/pubmed/15285897
Options:
-s Full path directory containing the splice5sequences and me2x5 splice model files.
See USeq/Documentation/ or http://genes.mit.edu/burgelab/maxent/download/
-t Full path file name for 9mer test sequences, GATCgatc only, one per line. Fasta OK.
Example: java -Xmx10G -jar pathTo/USeq/Apps/MaxEntScanScore5 -s ~/MES/splicemodels -t
~/MES/seqsToTest.fasta
**************************************************************************************
**************************************************************************************
** Merge Adjacent Regions: April 2022 **
**************************************************************************************
Merges regions within a max bp gap and tracks the number merged. Regions must not
overlap. Best run the MergeRegions app if in doubt.
Options:
-b Path to a bed file of non overlapping regions, xxx.gz/.zip OK.
-r Path for saving the merged xxx.bed.gz file.
-m Max bp gap, defaults to 5000.
Example: java -Xmx4G -jar pathTo/USeq/Apps/MergeAdjacentRegions -b myRegions.bed.zip
-m 1000 -r mergedRegions.bed.gz
**************************************************************************************
**************************************************************************************
** Merge Overlappng Genes: Feb 2015 **
**************************************************************************************
Merges transcript models that share exonic bps. Maximizes exons, minimizes introns.
Assumes interbase coordinates.
Options:
-u Path to a UCSC RefFlat or RefSeq gene table file or directory with such to merge.
See http://genome.ucsc.edu/cgi-bin/hgTables, (geneName name2(optional) chrom
strand txStart txEnd cdsStart cdsEnd exonCount exonStarts exonEnds).
-r Path for results file.
-m Minimum fraction exonic bp overlap for merging, defaults to 0.05
Example: java -Xmx4G -jar pathTo/USeq/Apps/MergeOverlappingGenes -d
/CufflinkTranscripts/zv9Genes.ucsc.gz -f 0.25 -r merged.ucsc
**************************************************************************************
**************************************************************************************
** Merge Paired Alignments: June 2020 **
**************************************************************************************
Merges proper paired alignments that pass a variety of checks and thresholds. Only
unambiguous pairs will be merged. Increases base calling accuracy in overlap and helps
avoid non-independent variant observations and other double counting issues. Identical
overlapping bases are assigned the higher quality scores. Disagreements are resolved
toward the higher quality base. If too close in quality, then the quality is set to 0.
Options:
-b Path to a coordinate sorted xxx.bam file containing paired alignments.
-d Path to a directory for saving the results.
Default Options:
-s Save merged xxx.sam.gz alignments instead of binary ChromData. Either works
in Sam2USeq for read coverage analysis, the ChromData is much faster.
-u Remove all alignments marked as duplicates, defaults to keeping.
-a Maximum alignment score (AS:i: tag). Defaults to 300, smaller numbers are more
stringent for novoalign where each mismatch is ~30pts.
-q Minimum mapping quality score, defaults to 0, larger numbers are more stringent.
-r The second paired alignment's strand has been reversed. Defaults to not reversed.
-i Maximum acceptible base pair distance for merging, defaults to 5000.
-m Don't cross check read mate coordinates, needed for merging repeat matches. Defaults
to checking.
-o Merge all proper paired alignments. Defaults to only merging those that overlap.
-p Don't print detailed paired alignment statistics and insert size histogram.
-t Number concurrent threads to run, defaults to the max available to the jvm.
-j Write summary stats in json format to this file.
Example: java -Xmx20G -jar pathToUSeq/Apps/MergePairedBamAlignments -f /Bams/ms.bam
-p -s /Bams/MergedPairs/ms.mergedPairs.sam.gz -d 10000
**************************************************************************************
**************************************************************************************
** Merge Point Data: Jan 2011 **
**************************************************************************************
Efficiently merges PointData, collapsing by position and possibly strand. Identical
position scores are either summed or converted into counts. DO NOT use this app on
PointData that will be part of a primary chIP/RNA-seq analysis. It is only for
bis-seq and visualization purposes.
Options:
-p Point Data directories, full path, comma delimited. Should contain chromosome
specific xxx.bar.zip or xxx_-_.bar files. Alternatively, provide one directory
containing multiple PointData directories.
-s Save directory, full path.
-c Don't replace scores with hit count, just sum existing scores.
-m Merge strands
Example: java -Xmx1500M -jar pathTo/USeq/Apps/MergePointData -p
/Data/Ets1Rep1/,/Data/Ets1Rep2/ -s /Data/MergedEts1 -m
**************************************************************************************
**************************************************************************************
** Merge Regions: Dec 2021 **
**************************************************************************************
Flattens tab delimited bed files (chr start stop ...). Assumes interbase coordinates.
Set the -m threshold to restrict the output to bps with that minimum of overlapping
regions.
Options:
-r Path to a region file or directory containing such, xxx.gz/xxx.zip OK.
-m Minimum overlapping bp to save, defaults to 1
-p Pad input regions +/- bps, defaults to 0
-o Path to the merged output file, optional
Example: java -Xmx4G -jar pathTo/Apps/MergeRegions -r PassingBedFiles/ -m 3 -p 150
************************************************************************************
**************************************************************************************
** Merge Sams: May 2017 **
**************************************************************************************
Merges sam and bam files. Adds a consensus header if one is not provided. These may
not work with GATK or Picard downstream apps, good for USeq.
Options:
-d The full path to a directory containing xxx.bam or xxx.sam.gz files to merged.
Default Options:
-s Save file, must end in xxx.bam, defaults merge.bam in -d.
-a Maximum alignment score. Defaults to 300, smaller numbers are more stringent.
Approx 30pts per mismatch.
-m Minimum mapping quality score, defaults to 0 (no filtering), larger numbers are
more stringent. Set to 13 or more to require near unique alignments. DO NOT set
for alignments parsed by the SamTranscriptomeParser!
-f Save reads failing filters, defaults to tossing them.
-h Full path to a txt file containing a sam header, defaults to autogenerating the
header from the sam/bam headers.
-t Don't delete temp xxx.sam.gz file.
-p Add program arguments to header, defaults to deleting, note duplicate cause Picard
apps to fail.
-q Quiet, print only errors.
Example: java -Xmx1500M -jar pathToUSeq/Apps/MergeSams -f /Novo/Run7/
-m 20 -a 120
**************************************************************************************
**************************************************************************************
** Merge UCSC Gene Table: Aug 2019 **
**************************************************************************************
Merges transcript models that share the same gene name(in column 0) and chromosome.
Maximizes exons, minimizes introns. Assumes interbase coordinates.
Options:
-u UCSC RefFlat or RefSeq gene table file, full path. See,
http://genome.ucsc.edu/cgi-bin/hgTables, (geneName name2(optional) chrom strand
txStart txEnd cdsStart cdsEnd exonCount exonStarts exonEnds).
Example: java -Xmx4G -jar pathTo/USeq/Apps/MergeUCSCGeneTable -u
/data/zv8EnsemblGenes.ucsc.gz
**************************************************************************************
**************************************************************************************
** Methylation Array Scanner: March 2014 **
**************************************************************************************
MAS takes paired or non-paired sample PointData representing beta values (0-1) from
arrays and scores regions with enriched/ reduced signal using a sliding window
approach. A B&H corrected Wilcoxon signed rank (or rank sum test for non-paired),
pseudo median of the log2(treat/control) ratios (or log2(pseT/pseC) for non-paired),
and permutation test FDR is calculated for each window. Use the EnrichedRegionMaker
to identify enriched and reduced regions by picking thresholds (e.g. -i 0,1 -s 0.2,13).
MAS generates several data tracks for visualization in IGB including paired sample bp
log2 ratios, window level Wilcoxon FDRs, and window level pseudomedian log2 ratios.
Note, non-paired analysis are very underpowered and require > 30 obs/ window to see
any significant FDRs.
Required Options:
-s Path to a directory for saving the results.
-d Path to a directory containing individual sample PointData directories, each of
which should contain chromosome split bar files (e.g. chr1.bar, chr2.bar, ...)
-t Names of the treatment sample directories in -d, comma delimited, no spaces.
-c Ditto but for the control samples, the ordering is critical and describes how to
pair the samples for a paired analysis.
Advanced Options:
-n Run a non-paired analysis where t and c are treated as groups and pooled.
-w Window size, defaults to 1000.
-o Minimum number observations in window, defaults to 10.
-p Minimum pseudomedian log2 ratio for estimating the permutation FDR, defaults to 0.2
-r Number permutations, defaults to 5
-e Run T-Test instead of Wilcoxon rank sum test for non-paired samples.
-v Save coefficient of variantion tracks
Example: java -Xmx4G -jar pathTo/USeq/Apps/MethylationArrayScanner -s ~/MAS/Res
-d ~/MAS/Bar/ -t Early1,Early2,Early3 -c Late1,Late2,Late3
-w 1500
**************************************************************************************
**************************************************************************************
** Methylation Array Defined Region Scanner: July 2013 **
**************************************************************************************
MADRS takes paired sample PointData representing beta values (0-1) from arrays and
a list of regions to score for differential methylation using a B&H corrected Wilcoxon
signed rank test and pseudo median of the paired log2(treat/control) ratios. Pairs
containing a zero value are ignored. It generates a spreadsheet of statistics for each
region. If a non-paired analysis is selected, a Wilcoxon rank sum test and
log2(pseT/pseC) are calculated on each region. Note this is a very underpowered test
requiring >30 observations to see any significant FDRs.
Required Options:
-b A bed file of regions to score (tab delimited: chr start stop ...)
-d Path to a directory containing individual sample PointData directories, each of
which should contain chromosome split bar files (e.g. chr1.bar, chr2.bar, ...)
-t Names of the treatment sample directories in -d, comma delimited, no spaces.
-c Ditto but for the control samples, the ordering is critical and describes how to
pair the samples for a paired analysis.
-o Minimum number paired observations in window, defaults to 3.
-z Skip printing regions with less than minimum observations.
-n Run a non-paired analysis where t and c are treated as groups and pooled. Uneven
numbers of t and c are allowed.
Example: java -Xmx4G -jar pathTo/USeq/Apps/MethylationArrayDefinedRegionScanner
-v H_sapiens_Feb_2009 -d ~/MASS/Bar/ -t Early1,Early2,Early3
-c Late1,Late2,Late3
**************************************************************************************
**************************************************************************************
** Microsatellite Counter: Jan 2014 **
**************************************************************************************
MicrosatelliteCounter identifies and counts microsatellite repeats in MiSeq fastq
files. This iteration of the software requires you to specify the primers used in the
sequencing project. It will automatically find the most likely microsatellite by
looking at all possible repeats of length 1 through length 10 and finding the longest
repeat by length, not repeat unit. There are two output files generated, the first
lists primer statistics (currently only reads with both primers are used), the
second lists repeat data. Note that the input file are fastq sequence that were
merged using a program like PEAR
Required Arguments:
-f Merged fastq file. Path to merged fastq file. We currently suggest using PEAR to
merge fastq sequences.
-p Primer file. Path to primer reference file. This file lists each primer used in
in the sequencing project in the format NAME
**************************************************************************************
** MiRNA Correlator: March 2014 **
**************************************************************************************
Generates a spreadsheet to use in comparing changing miRNA levels to changes in gene
expression.
Options:
-r Results file.
-a All miRNA name file (single column of miRNA names).
-m MiRNA data (two columns: miRNA name, miRNA log2Rto).
-t Gene target to miRNA data (two columns: gene target name, miRNA name).
-e Gene expression data (three columns: gene name, log2Rto, FDR).
-f Don't print the gene expression FDR value in the spreadsheet.
Example: java -Xmx4G -jar pathTo/USeq/Apps/MiRNACorrelator -m miRNA_CLvsMOR.txt -a
allMiRNANamesNoPs.txt -t targetGene2MiRNA.txt -e geneExp_CLvsMOR.txt -r results.xls
**************************************************************************************
**************************************************************************************
** MpileUp Parser: April 2015 **
**************************************************************************************
Parses a SAMTools mpileup output file for non reference bases generating bed files and
data tracks with information related to error prone bases. Multiple samples are merged.
Options:
-p Path to a mpileup file (.gz or.zip OK, use 'samtools mpileup -Q 20 -A -B *bam').
-v Versioned Genome (ie H_sapiens_Mar_2006), see UCSC Browser,
http://genome.ucsc.edu/FAQ/FAQreleases.
-s Save directory, full path, defaults to pileup file directory.
-r Minimum read coverage, defaults to 15.
-e Max nonRef base fraction to score as a failing bp, defaults to 0.01
-w Window size, defaults to 50
-f Max fraction failing bp in window to score as a high error window, defaults to 0.05
Example: java -Xmx4G -jar pathTo/USeq/Apps/MpileupParser -p /Pileups/N2.mpileup.gz -v
H_sapiens_Feb_2009 -e 0.1 -w 25
**************************************************************************************
**************************************************************************************
** Mpileup Randomizer: May 2018 **
**************************************************************************************
Upon finding a gap in the coverage, the sample order is randomized and maintained. Use
this app to 'de-identify' a multi sample mpileup file while maintaining INDEL blocks.
Required Options:
-m Path to a Samtools mpileup file (gz/zip OK).
Default Options:
-r Minimum read depth to pass a sample, default 10
-s Minimum number of samples that must pass to save line, default 3
-g Minimum gap, defaults to 125
Example: java -Xmx4G -jar pathToUSeq/Apps/MpileupRandomizer -m normExo.mpileup.gz
-r 20 -s 4
**************************************************************************************
**************************************************************************************
** Multiple Replica Scan Seqs: May 2019 **
**************************************************************************************
MRSS uses a sliding window and Love et al's DESeq2 negative binomial -> Benjamini &
Hochberg AdjP statistics to identify enriched and reduced regions in a genome. Both
treatment and control PointData sets are required, two or more biological replicas.
MRSS generates window level differential count tracks for the AdjP and normalized
log2Ratio as well as a binary window objec xxx.swi file for downstream use by the
EnrichedRegionMaker. MRSS also makes use of DESeq2's variance corrected count data to
cluster your biological replics. Given R's poor memory management, running DESeq2
requires lots of RAM, 64bit R, and patience.
Options:
-s Save directory, full path.
-t Treatment replica PointData directories, full path, comma delimited, no spaces,
one per biological replica. Use the PointDataManipulator app to merge same
replica and technical replica datasets. Each directory should contain stranded
chromosome specific xxx_-/+_.bar.zip files. Alternatively, provide one
directory that contains multiple biological replical PointData directories.
-c Control replica PointData directories, ditto.
-r Full path to 64bit R loaded with DESeq library, defaults to '/usr/bin/R' file, see
http://www-huber.embl.de/users/anders/DESeq/ . Type 'library(DESeq2)' in
an R terminal to see if it is installed.
-p Peak shift, average distance between + and - strand peaks for chIP-Seq data, see
PeakShiftFinder or set it to 100bp. For RNA-Seq set it to 0. It will be used
to shift the PointData by 1/2 the peak shift.
-w Window size, defaults to the peak shift. For chIP-Seq data, a good alternative
is the peak shift plus the standard deviation, see the PeakShiftFinder app.
For RNA-Seq data, set this to 100-250.
Advanced Options:
-m Minimum number of reads in a window, defaults to 15
-d Don't delete temp files
Example: java -Xmx4G -jar pathTo/USeq/Apps/MultipleReplicaScanSeqs -t
/Data/PolIIRep1/,/Data/PolIIRep2/ -c /Data/Input1/,Data/Input2/ -s
/Data/PolIIResults/ -p 150 -w 250 -b
**************************************************************************************
**************************************************************************************
** Multi Sample VCF Filter : July 2015 **
**************************************************************************************
Filters a vcf file containing multiple sample records into those that pass or fail the
tests below. This works with VCFv4.1 files created by the GATK package. Note, the
records are not modified. If the number of records in the VCF file is greater than
500000, the VCF file is intersected in chunks.
Required:
-v Full path to a sorted single or multi sample vcf file (xxx.vcf/xxx.vcf.gz)). Note,
Java often fails to parse tabix compressed vcf files. Best to uncompress.
Optional:
-p Full path to an output VCF (xxx.vcf or xxx.vcf.gz). Specifying xxx.vcf.gz will
compress and index the VCF using tabix (set -t too). Defaults to input_Filt.vcf
-f Print out failing records, defaults to printing those passing the filters.
-a Fail records where no sample passes the sample thresholds.
-i Fail records where the original FILTER field is not 'PASS' or '.'
-c Fail records that don't intersect the regions in this bed file, full path.
-b Filter by genotype flags. -n, -u and -l must be set.
-n Sample names ordered by category.
-u Number of samples in each category.
-l Requirement flags for each category. All samples that pass the specfied filters
must meet the flag requirements, or the variant isn't reported. At least one
sample in each group must pass the specified filters, or the variant isn't
reported.
a) 'W' : homozygous common
b) 'H' : heterozygous
c) 'M' : homozygous rare
d) '-W' : not homozygous common
e) '-H' : not heterozygous
f) '-M' : not homozygous rare
-e Strict genotype matching. If this is selected, records with no-call samples
or samples falling below either minimum sample genotype quality (-g) or
minimum sample read depth (-r) won't be reported. Only samples listed in (-n)
will be checked
-d Minimum record QUAL score, defaults to 0, recommend >=20
-g Minimum sample genotype quality GQ, defaults to 0, recommend >= 20
-r Minimum sample read depth DP, defaults to 0, recommend >=10
-x Maximum sample read depth DP, defaults to unlimited
-y Minimum sample allele count read depth AD or DP4, defaults to 0
-s Print sample names and exit.
-t Path to tabix.
Example: java -Xmx10G -jar pathTo/USeq/Apps/MultiSampleVCFFilter
-v DEMO.passing.vcf -p DEMO.intersection.vcf -c exomeV4.bed -b
-n SRR504516,SRR776598,SRR504515,SRR504517,SRR504483 -u 2,2,1 -l M,H,-M
**************************************************************************************
**************************************************************************************
** Mutect 4 VCF Parser: March 2021 **
**************************************************************************************
Parses Mutect2 VCF files from the GATK 4.0+ package and Sentieon TNhaplotyper2 for
read depth, allele frequency diff ratio, etc. Inserts AF and DP into for the tumor
sample into the INFO field. Replaces the QUAL with TLOD. Be sure to run vt
normalization first and check the sample order.
Options:
-v Full path file or directory containing xxx.vcf(.gz/.zip OK) file(s). It is REQUIRED
to run 'vt decompose -s ' on these first. Recommend running decompose_blocksub
too. See https://github.com/atks/vt
-f Directory to save the parsed files, defaults to the parent dir of the first vcf.
-t Minimum tumor allele frequency (AF), defaults to 0.
-n Maximum normal AF, defaults to 1.
-u Minimum tumor alignment depth, defaults to 0.
-a Minimum tumor alt count, defaults to 0.
-o Minimum normal alignment depth, defaults to 0.
-d Minimum T-N AF difference, defaults to 0.
-r Minimum T/N AF ratio, defaults to 0.
-t Minimum TLOD score, defaults to 0.
-p Remove non PASS filter field records.
-x Sample order is Tumor then Normal, defaults to Normal then Tumor. Check!
Example: java -jar pathToUSeq/Apps/Mutect4VCFParser -v /VCFFiles/ -t 0.06 -n 0.5 -u 100
-o 20 -d 0.05 -r 2 -a 3
**************************************************************************************
**************************************************************************************
** Non LoH : Nov 2022 **
**************************************************************************************
LoH compares heterozygous snv and indel allele counts between normal and somatic
sequencing datasets to identify potential loss of heterozygosity events. LoH is
defined here as a significant increase in the somatic allele fraction (AF) relative to
the matched normal. Adjacent variants passing p-value and AF difference thresholds
are merged and a combine p-value calculated for the window block.
Required:
-v Path to a normal, single sample, variant xxx.vcf(.gz/.zip OK) file. These should
be filtered, high confidence, vt normalized and decomposed short variants.
-g Path to the normal bampileup file, use the USeq BamPileup tool to generate the
xxx.bp.txt.gz and tabix index files.
-s Path to the somatic bampileup file, ditto.
-p Path to a directory containing normal sample bampileup files for the PoN
-o Path to directory to write the bed and vcf output result files.
Optional:
-c Bed file containing passing regions from the TNRunner2 CopyRatio workflow.
-q Minimum vcf record QUAL, defaults to 20
-e Minimum vcf record sample GQ, defaults to 20
-x Use Strelka's recalibrated GQX genotype score, defaults to GQ
-r Minimum unique observation variant read depth for both normal and tumor samples,
defaults to 20
-l Minimum Fisher exact p-value for including a variant in a block, defaults to 0.1
-f Minimum difference in AF between the somatic and normal samples for including
a variant into a block, defaults to 0.1
-m Maximum bp gap between variants for grouping into a block, defaults to 1000
-p Minimum Benjamini-Hochberg adjusted window -10Log10(combine p-value), defaults to
8.239 (0.15)
-d Minimum window mean difference in AFs, defaults to 0.1
-w Window BP padding for reporting significant LoH regions, defaults to 100
Example: java -Xmx4G -jar pathTo/USeq/Apps/LoH -v gatkStrelkaNormal.vcf.gz
-g normal.bp.txt.gz -s tumor.bp.txt.gz -o LoHResults/ -d 0.15 -m 5000 -b
GATKCopyRatio_Hg38.called.seg.pass.bed.gz
**************************************************************************************
**************************************************************************************
** Non Reference Region Maker: Jan 2018 **
**************************************************************************************
NRRM scans a single sample mpileup file looking for non reference base pairs. If these
pass read depth, allele frequency, and non ref base count thresholds, the base is
written to a bed file. BPs with insertions are saved as a 2 BP region. Run MergeRegions
or MergeAdjacentRegions to join proximal non ref BPs.
Options:
-m Provide a path to a single sample samtools mpileup file or pipe mpileup output.
-b Path to write the bed file output, should end in xxx.bed.gz
-r Minimum read depth, 10
-a Minimum non reference allelic frequency (SNVs + INDELS), default 0.05
-c Minimum non reference base count, default 3
-q Minimum base quality for inclusion in AF calculation, default 10
Example: samtools mpileup -B -d 1000000 -f $faIndex -l $bed $bam | java
-Xmx4G -jar pathToUSeq/Apps/NonReferenceRegionMaker -q 13 -r 20 -a 0.025 -b
0.025normExoNonRefMask.bed.gz -c 4
**************************************************************************************
**************************************************************************************
** Novoalign Bisulfite Parser: May 2016 **
**************************************************************************************
Parses Novoalign -b2 and -b4 single and paired bisulfite sequence alignment files into
PointData file formats. Generates several summary statistics on converted and non-
converted C contexts. Flattens overlapping reads in a pair to call consensus bps.
Note: for paired read RNA-Seq data run through the SamTranscriptomeParser first.
Options:
-a Alignment file or directory containing non merged novoalignments in SAM/BAM
(xxx.sam(.zip/.gz OK) or xxx.bam) format. Multiple files are combine.
-f Fasta file directory, chromosome specific xxx.fa/.fasta(.zip/.gz OK) files.
-s Save directory.
-v Versioned Genome (ie H_sapiens_Mar_2006), see UCSC Browser,
http://genome.ucsc.edu/FAQ/FAQreleases.
Default Options:
-p Print bed file parsed data.
-x Maximum alignment score. Defaults to 300, smaller numbers are more stringent.
-q Minimum mapping quality score. Defaults to 13, bigger numbers are more stringent.
This is a phred-scaled posterior probability that the mapping position of read
is incorrect. For RNASeq data, set this to 0.
-b Minimum base quality score for reporting a non/converted C, defaults to 13.
-c Minimum base quality score for reporting a overlapping non/converted C not found
in the other pair, defaults to 13.
-d Remove duplicate reads prior to generating PointData. Defaults to not removing
duplicates.
Example: java -Xmx25G -jar pathToUSeq/Apps/NovoalignBisulfiteParser -x 240 -a
/Novo/Run7/ -f /Genomes/Hg19/Fastas/ -v H_sapiens_Feb_2009 -s /Novo/Run7/NBP
**************************************************************************************
**************************************************************************************
** Novoalign Indel Parser: June 2010 **
**************************************************************************************
Parses Novoalign alignment xxx.txt(.zip/.gz) files for consensus indels, something
currently not supported by the maq apps. Generates a consensus indel allele file,
interbase coordinates, for running through the Alleler application. Also creates two
bed files for the insertions and deletions.
Options:
-f The full path directory/file text of your Novoalign xxx.txt(.zip or .gz) file(s).
-r Full path directory for saving the results.
-p Minimum alignment posterior probability (-10Log10(prob)) of being incorrect,
defaults to 13 (0.05). Larger numbers are more stringent.
-b Minimum effected indel base quality score(s), ditto, defaults to 13.
-u Minimum number of unique reads covering indel, defaults to 2.
Example: java -Xmx1500M -jar pathToUSeq/Apps/NovoalignIndelParser -f /Novo/Run7/
-r /Novo/Run7/indelAlleleTable.txt -p 20 -b 20 -u 3
**************************************************************************************
**************************************************************************************
** Novoalign Parser: Jan 2011 **
**************************************************************************************
Parses Novoalign xxx.txt(.zip/.gz) files into center position binary PointData xxx.bar
files, xxx.bed files, and if appropriate, a splice junction bed file. For the later,
create a gene regions bed file and run it through the MergeRegions application to
collapse overlapping transcripts. We recommend using the following settings while
running Novoalign 'novoalign -r0.2 -q5 -d yourDataBase -f your_prb.txt | grep '>chr' >
yourResultsFile.txt'. NP works with native, colorspace, and miRNA novoalignments.
Options:
-v Versioned Genome (ie H_sapiens_Mar_2006), see UCSC Browser,
http://genome.ucsc.edu/FAQ/FAQreleases.
-f The full path directory/file text of your Novoalign xxx.txt(.zip or .gz) file(s).
-r Full path directory text for saving the results.
-p Posterior probability threshold (-10Log10(prob)) of being incorrect, defaults to 13
(0.05). Larger numbers are more stringent. The parsed scores are delogged and
converted to 1-prob.
-q Alignment score threshold, smaller numbers are more stringent, defaults to 60
-c Chromosome prefix, defaults to '>chr'.
-i Ignore strand when making splice junctions.
-g (Optional) Full path gene region bed file (chr start stop...) containing gene
regions to use in scaling intersecting splice junctions.
-s Just print alignment stats, don't save any data.
Example: java -Xmx1500M -jar pathToUSeq/Apps/NovoalignParser -f /Novo/Run7/
-v H_sapiens_Mar_2006 -p 20 -q 30 -r /Novo/Run7/mRNASeq/ -i -g
/Anno/Hg18/mergedUCSCKnownGenes.bed
**************************************************************************************
**************************************************************************************
** Novoalign Paired Parser: January 2009 **
**************************************************************************************
Parses Novoalign paired alignment files xxx.txt(.zip/.gz) into xxx.bed format.
Options:
-f The full path directory/file text of your Novoalign xxx.txt(.zip or .gz) file(s).
-e Exclude half matches with a high quality unmatched pair, defaults to keeping them.
-m Maximum size for paired reads mapping to the same chromosome, defaults to 100000.
-s Splice junction radius, defaults to 34. See the MakeSpliceJunctionFasta app.
Example: java -Xmx1500M -jar pathToUSeq/Apps/NovoalignPairedParser -f /Novo/Run7/
**************************************************************************************
**************************************************************************************
** Overdispersed Region Scan Seqs: May 2012 **
**************************************************************************************
WARNING: this application is depreciated and no longer maintained, use the
DefinedRegionDifferentialSeq app instead!
ORSS takes bam alignment files and extracts reads under each region or gene's exons to
calculate several statistics. Makes use of Simon Anders' DESeq R package to with its
negative binomial p-value test to control for overdispersion. A Benjamini-Hochberg FDR
correction is used to control for multiple testing. DESeq is run with and without
variance outlier filtering. A chi-square test of independence between the exon read
count distributions is used to score alternative splicing. Several read count measures
are provided including counts for each replica, FPKMs (# frags per kb of int region
per total mill mapped reads) as well as DESeq's variance adjusted counts(use these for
clustering, correlation, and other distance type analysis). If replicas are provided
either the smallest all pair log2Ratio is reported (default) or the pseudomedian.
Several results files are written: two spread sheets containing all of the genes,
those that pass the thresholds, as well as egr, bed12, and useq region files for
visualization in genome browsers.
Required Options:
-s Save directory.
-t Treatment directory containing one xxx.bam file with xxx.bai index per biological
replica. The BAM files should be sorted by coordinate and have passed Picard
validation. Use the SamTranscriptomeParser to convert your aligned transcriptome
data to genomic coordinates.
-c Control directory, ditto.
-u UCSC RefFlat or RefSeq Gene table file, full path. See,
http://genome.ucsc.edu/cgi-bin/hgTables, (name1 name2(optional) chrom strand
txStart txEnd cdsStart cdsEnd exonCount exonStarts exonEnds). WARNING!!!!!!
This table should contain only one composite transcript per gene. Use the
MergeUCSCGeneTable app to collapse Ensembl transcripts downloaded from UCSC in
RefFlat format.
-b (Or) a bed file (chr, start, stop,...), full path, See,
http://genome.ucsc.edu/FAQ/FAQformat#format1
-v Versioned Genome (ie H_sapiens_Mar_2006, D_rerio_Jul_2010), see UCSC Browser,
http://genome.ucsc.edu/FAQ/FAQreleases.
Advanced/ Default Options:
-o Don't remove overlapping exons, defaults to filtering gene annotation for overlaps.
-i Score introns instead of exons.
-a Data is stranded. Only collect reads from the same strand as the annotation.
-f Minimum FDR threshold, defaults to 10 (-10Log10(FDR=0.1))
-l Minimum absolute log2 ratio threshold, defaults to 1 (2x)
-e Minimum number mapping reads per region, defaults to 20
-d Don't delete temp files used by DESeq
-p Use a pseudo median log2 ratio in place of the smallest all pair log2 ratios for
scoring the degree of differential expression when replicas are present.
Recommended for experiments with 4 or more replicas.
-r Full path to R loaded with DESeq library, defaults to '/usr/bin/R' file, see
http://www-huber.embl.de/users/anders/DESeq/ . Type 'library(DESeq)' in
an R terminal to see if it is installed.
Example: java -Xmx4G -jar pathTo/USeq/Apps/OverdispersedRegionScanSeqs -t
/Data/PolIIRep1/,/Data/PolIIRep2/ -c /Data/Input1/,Data/Input2/ -s
/Data/PolIIResults/ -f 30 -e 30 -u /Anno/mergedZv9EnsemblGenes.ucsc.gz
**************************************************************************************
**************************************************************************************
** ParseIntersectingAlignments: June 2010 **
**************************************************************************************
Parses bed alignment files for intersecting reads provided another bed file of alleles.
Options:
-s Full path file text for your SNP allele five column bed file (tab delimited chr,
start,stop,text,score,strand)
-a Full path file text for your alignment bed file from the NovoalignParser.
-m Minimum base quality, defaults to 13
Example: java -Xmx1500M -jar pathToUSeq/Apps/ParseIntersectingAlignments
-s /LympAlleles/ex1.bed -a /SeqData/lymphAlignments.bed -m 13
**************************************************************************************


Matt Starkey
01.19.17
If she were brought up in just about any other family in the country, Amani Burke would be an outside hitter for a Division I volleyball team, playing on full scholarship. Her mother, Ronda, is sure of it.
In the right setting, she’d probably make a great soccer player, too. Ronda said that as a credit not only to the athleticism Amani possesses, but also to her intellect when it comes to sports. If there’s a game to be played, it likely won’t take long for the Columbus native to excel at it.
But as fate would have it, Amani was born a Burke, the daughter of a former Northwest Missouri State basketball player, Alfonso II — or “Benji” — and the younger sister of a current Washington Wizards point guard, Trey. It was always going to be basketball for Amani, and halfway through her freshman season at Ohio, she’s already establishing herself as an important piece of the next wave of talent in one of the Mid-American Conference’s strongest programs.
“I think the transition is still happening,” Amani said. “It’s not as hard as I thought it would be. It’s exciting being able to come off the bench and play as often as I do.”
Basketball called to each of the Burke children by name, one by one.
The first call was also the most brief. Amber, the oldest child, began playing the sport at a young age, but a shot to the mouth during a middle school game was all the reason she needed to stop playing basketball, opting for less contact-filled pursuits, such as cheerleading and dancing.
Then came Alfonso III, better known as Trey, the astonishingly talented point guard who by 5 years old was dominating competitors to the point the league rewrote its rules just to keep him under control.
Amani Burke poses with her brother, Trey Burke, and their family at the 2013 NBA draft. (Provided via Ronda Burke)
Trey was soon playing for some of the top Amateur Athletic Union teams in the state, under the coaching of his father, beginning in second grade. And at every game, sitting on the bench with them would be Amani, five years younger than Trey, quietly absorbing everything she saw.
“Before she even started playing, she was a watcher,” Benji said. “She was at all of Trey’s AAU games and high school games, so she got a chance to really see a high level. … She was like a student. When she started playing, it was kind of weird because she didn’t look like a beginner player. She was mimicking a lot of things she’d already seen.”
Amani played basketball for the first time in fourth grade, just as Trey was beginning his ascent at Northland High School. From the beginning, however, it was apparent that the younger player approached the game much differently than the older one.
In his younger playing days, Trey always had great difficulty accepting bad games, often dwelling on poor performances and losses for a week or longer – his competitive spirit consuming him whole. Amani, on the other hand, always had a calmness to her both on and off the court, seeing the positives in any negative outcome.
“Don’t get me wrong, Amani takes it hard, but Trey was ridiculous,” Ronda said. “He’s extremely, extremely competitive. Amani goes out there to win the game, but it’s really water under the bridge (after losses) when it came time for the next one. Trey had to develop that skill a little more. Both of them have a really strong work ethic.”
As Trey’s fame continued to grow, Amani maintained her own sense of identity. Instead of trying to copy the game of her brother and his teammates, Amani developed her own style of play – one built around a philosophy that involves the entire offense and allowing her moments in the sun to occur naturally.
In the fall of 2012, as Trey began his second and final season at Michigan, Amani decided against enrolling at the high school at which her brother achieved nationwide fame. Instead of attending Northland High School, Amani opted to go to Eastmoor Academy. From there, she rewrote record books on her own, finishing as the program’s all-time leading scorer, leading her team to a final four appearance and earning Division II co-Player of the Year honors in 2016.
Left image provided by Wikimedia Commons. Right image was taken by Post photographer Matt Starkey
It was an impressive run for someone who, late into her high school career, still wanted to play volleyball in college instead of basketball. It took an extra push from her parents to convince her to stick to basketball full time.
“My opinion was that if she gave up basketball to pursue volleyball full-time, she would miss basketball,” Benji said. “Where if you reversed that, I didn’t think she would really miss volleyball. So I said you’ve got to be careful what you pick. I believe she was just as good at volleyball, but I thought basketball had a slight edge based on her talent.”
Teams across the country took notice of Amani’s basketball performances, and suddenly the Burkes were in the heat of a recruiting process for the second time. And while the recruiting process can be an incredibly trying time for an athlete and his or her family, the Burkes went into Amani’s recruiting with some wisdom they didn’t possess the first time around. Amani and her parents knew how to tell which colleges were serious about their interest.
Amani Burke signs on to play basketball at OU. (Provided via Ronda Burke)
“Our motto in this house is, ‘Love who loves you,’ ” Ronda said. “That’s how Trey ended up at Michigan, and that’s how Amani ended up at OU. There were other schools talking to her, but OU was on her hard.”
Ohio head coach Bob Boldon impressed the family with his interest in the player Amani already was, and not the one she could one day become.
“She’s very versatile,” Boldon said. “She’s made plays around the rim, and we’ve seen her shoot the three and post up a little bit to rebound and guard well. She does a lot of the little things we ask for out of our recruits. Then once you meet her and her family, they’re just a wonderful family. She’s just a perfect fit for us.”
The fit has turned out to be mutual. Amani knew she had a shot to earn playing time right away, which, combined with Ohio’s close proximity to home, made the school the most sensible landing spot.
“I knew that I could play with these girls,” Amani said. “Over the summer, I was trying to make sure I was first in sprints, doing extra workouts, trying to get stronger, trying to get faster. (Boldon) told me when I was being recruited that I was gonna be able to play, so I just took that and ran with it.”
The hour-long drive between Columbus and The Convo isn’t the only reason Amani has to feel close to home. Many of her Bobcat teammates, including Jasmine Weatherspoon, Quiera Lampkins and Yamonie Jenkins, played on AAU teams with Amani while she was growing up. Meanwhile, Tia Jameson, Amani’s coach during her freshman and sophomore years of high school, currently serves as Ohio’ Athletics’ Director of Operations.
Perhaps it’s that comfort factor that helped Amani — who averages 6.1 points and 2.4 rebounds in 16.4 minutes per game — adapt at a quick rate to the college game. However, maybe it’s the more demanding aspects of her transition that deserve the credit.
Amani Burke stands with her teammates during the National Anthem prior to a game against Kent State in The Convo on Saturday, Jan. 14.
In practices, Amani is tasked with guarding — and being guarded by — Lampkins – the reigning MAC Defensive Player of the Year, who is averaging 19.2 points per game in 2016. The challenge has generated a lot of hype about the expectations coaches might have for the freshman guard. But because she managed to stay out of her older brother’s shadow in Columbus, maybe she’ll continue casting her own light in Athens.
“She’s heard a lot that she can be the next Kiki (Lampkins),” Ronda said. “And Kiki is amazing. But I tell her she could also be the first Amani. She liked the sound of that, too.”
Left image provided by Wikimedia Commons. Right image was taken by Post photographer Matt Starkey


Registry サーバーのデプロイ
読む時間の目安: 16 分このページでは、オープンソースである Docker Registry を使って、独自にレジストリを提供する情報を示しています。 Docker Hub はホスト提供込みの Registry であり、さまざまな機能を追加することができます。 たとえばチーム、組織、ウェブフック、自動ビルドなどです。 詳しくは Docker Hub を参照してください。
Registry をデプロイするには、まずホスト上に Docker をインストールする必要があります。
Registry は registry
イメージのインスタンスの 1 つであり、Docker の内部において稼動します。
このトピックでは Registry のデプロイと設定方法に関する基本的な情報を示します。 設定オプションの全一覧については 設定リファレンス を参照してください。
外部から隔離された（air-gapped）データセンターを利用している場合は、隔離されたデータセンターでの配慮 を参照してください。
ローカル Registry の起動
以下のようなコマンドを実行して Registry コンテナーを起動します。
$ docker run -d -p 5000:5000 --restart=always --name registry registry:2
Registry がこうして利用できるようになります。
警告: ここに例としていくつか示す Registry の設定は、テスト環境向けにしか考慮していません。 本番環境向けの Registry であれば、TLS によって保護することが必要であり、理想的にはアクセス制御システムの利用が求められます。 ここでの説明を読み終えたら、本番環境向け Registry をデプロイするための 設定ガイド に進んでください。
Docker Hub から Registry へのイメージコピー
イメージは Docker Hub からプルして、それを Registry にプッシュすることができます。
以下の例では Docker Hub から ubuntu:16.04
イメージをプルして、そこに my-ubuntu
というタグをつけ直します。
そしてローカルな Registry にプッシュします。
その後にローカル上から ubuntu:16.04
と my-ubuntu
のイメージを削除した上で、ローカル Registry から my-ubuntu
イメージをプルします。
-
Docker Hub から
ubuntu:16.04
イメージをプルします。$ docker pull ubuntu:16.04
-
イメージに
localhost:5000/my-ubuntu
というタグをつけます。 このようにすることで、既存のイメージに追加のタグがつけられます。 タグ名のはじめにあるのがホスト名とポート番号です。 Docker はプッシュ処理時に、その名前部分を Registry の場所として解釈します。$ docker tag ubuntu:16.04 localhost:5000/my-ubuntu
-
localhost:5000
として起動しているローカルの Registry にイメージをプッシュします。$ docker push localhost:5000/my-ubuntu
-
ローカルにキャッシュされている
ubuntu:16.04
とlocalhost:5000/my-ubuntu
イメージを削除します。 この後には、ローカル Registry からイメージをプルしてみることができます。 Registry からはlocalhost:5000/my-ubuntu
イメージは削除されません。$ docker image remove ubuntu:16.04 $ docker image remove localhost:5000/my-ubuntu
-
ローカルの Registry から
localhost:5000/my-ubuntu
イメージをプルします。$ docker pull localhost:5000/my-ubuntu
ローカル Registry の停止
Registry を停止するには、コンテナーの場合と同じように docker container stop
コマンドを実行します。
$ docker container stop registry
コンテナーを削除するには docker container rm
を実行します。
$ docker container stop registry && docker container rm -v registry
基本的な設定
コンテナーを設定するには、docker run
コマンドの実行時に、追加のまたは修正のオプションを指定します。
以下の節では、Registry 設定に関する基本的なガイドラインを示します。 詳しくは Registry 設定リファレンス を参照してください。
Registry の自動起動
常用するインフラストラクチャーの一部として Registry を利用したい場合には、Docker の再起動時あるいは Registry 終了時に、Registry を自動的に再起動することが必要になります。
以下の例では --restart always
フラグを用いて、Registry に対して再起動ポリシーを設定します。

以下の例では Registry をポート 5001 で起動し、同時にその名称を registry-test
とします。
ここで -p
フラグの値の 1 つめはホストのポートであり、2 つめはコンテナー内のポートです。
コンテナーの内部にて Registry は、ポート 5000
を利用するのがデフォルトです。

以下のコマンドは、コンテナーの内部にて Registry はポート 5001 を待ち受けるようになります。
$ docker run -d \
-e REGISTRY_HTTP_ADDR=<IP_ADDRESS>:5001 \
-p 5001:5001 \
--name registry-test \
registry:2
ストレージの設定
ストレージ保存場所の変更
By default, your registry data is persisted as a docker volume
on the host filesystem. If you want to store your registry contents at a specific
location on your host filesystem, such as if you have an SSD or SAN mounted into
a particular directory, you might decide to use a bind mount instead. A bind mount
is more dependent on the filesystem layout of the Docker host, but more performant
in many situations. The following example bind-mounts the host directory
/mnt/registry
into the registry container at /var/lib/registry/
.
$ docker run -d \
-p 5000:5000 \
--restart=always \
--name registry \
-v /mnt/registry:/var/lib/registry \
registry:2
Customize the storage back-end
By default, the registry stores its data on the local filesystem, whether you use a bind mount or a volume. You can store the registry data in an Amazon S3 bucket, Google Cloud Platform, or on another storage back-end by using storage drivers. For more information, see storage configuration options.
Run an externally-accessible registry
Running a registry only accessible on localhost
has limited usefulness. In
order to make your registry accessible to external hosts, you must first secure
it using TLS.
This example is extended in Run the registry as a service below.
Get a certificate
These examples assume the following:
- Your registry URL is
https://myregistry.domain.com/
. - Your DNS, routing, and firewall settings allow access to the registry’s host on port 443.
- You have already obtained a certificate from a certificate authority (CA).
If you have been issued an intermediate certificate instead, see use an intermediate certificate.
-
Create a
certs
directory.$ mkdir -p certs
Copy the
.crt
and.key
files from the CA into thecerts
directory. The following steps assume that the files are nameddomain.crt
anddomain.key
. -
Stop the registry if it is currently running.
$ docker container stop registry
-
Restart the registry, directing it to use the TLS certificate. This command bind-mounts the
certs/
directory into the container at/certs/
, and sets environment variables that tell the container where to find thedomain.crt
anddomain.key
file. The registry runs on port 443, the default HTTPS port.$ docker run -d \ --restart=always \ --name registry \ -v "$(pwd)"/certs:/certs \ -e REGISTRY_HTTP_ADDR=<IP_ADDRESS>:443 \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \ -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \ -p 443:443 \ registry:2
-
Docker clients can now pull from and push to your registry using its external address. The following commands demonstrate this:
$ docker pull ubuntu:16.04 $ docker tag ubuntu:16.04 myregistry.domain.com/my-ubuntu $ docker push myregistry.domain.com/my-ubuntu $ docker pull myregistry.domain.com/my-ubuntu
Use an intermediate certificate
A certificate issuer may supply you with an intermediate certificate. In this
case, you must concatenate your certificate with the intermediate certificate to
form a certificate bundle. You can do this using the cat
command:
cat domain.crt intermediate-certificates.pem > certs/domain.crt
You can use the certificate bundle just as you use the domain.crt
file in
the previous example.
Support for Let’s Encrypt
The registry supports using Let’s Encrypt to automatically obtain a browser-trusted certificate. For more information on Let’s Encrypt, see https://letsencrypt.org/how-it-works/ and the relevant section of the registry configuration.
Use an insecure registry (testing only)
It is possible to use a self-signed certificate, or to use our registry insecurely. Unless you have set up verification for your self-signed certificate, this is for testing only. See run an insecure registry.
Run the registry as a service
Swarm services provide several advantages over standalone containers. They use a declarative model, which means that you define the desired state and Docker works to keep your service in that state. Services provide automatic load balancing scaling, and the ability to control the distribution of your service, among other advantages. Services also allow you to store sensitive data such as TLS certificates in secrets.
The storage back-end you use determines whether you use a fully scaled service or a service with either only a single node or a node constraint.
-
If you use a distributed storage driver, such as Amazon S3, you can use a fully replicated service. Each worker can write to the storage back-end without causing write conflicts.
-
If you use a local bind mount or volume, each worker node writes to its own storage location, which means that each registry contains a different data set. You can solve this problem by using a single-replica service and a node constraint to ensure that only a single worker is writing to the bind mount.
The following example starts a registry as a single-replica service, which is accessible on any swarm node on port 80. It assumes you are using the same TLS certificates as in the previous examples.
First, save the TLS certificate and key as secrets:
$ docker secret create domain.crt certs/domain.crt
$ docker secret create domain.key certs/domain.key
Next, add a label to the node where you want to run the registry.
To get the node’s name, use docker node ls
. Substitute your node’s name for
node1
below.
$ docker node update --label-add registry=true node1
Next, create the service, granting it access to the two secrets and constraining
it to only run on nodes with the label registry=true
. Besides the constraint,
you are also specifying that only a single replica should run at a time. The
example bind-mounts /mnt/registry
on the swarm node to /var/lib/registry/
within the container. Bind mounts rely on the pre-existing source directory,
so be sure /mnt/registry
exists on node1
. You might need to create it before
running the following docker service create
command.
By default, secrets are mounted into a service at /run/secrets/<secret-name>
.
$ docker service create \
--name registry \
--secret domain.crt \
--secret domain.key \
--constraint 'node.labels.registry==true' \
--mount type=bind,src=/mnt/registry,dst=/var/lib/registry \
-e REGISTRY_HTTP_ADDR=<IP_ADDRESS>:443 \
-e REGISTRY_HTTP_TLS_CERTIFICATE=/run/secrets/domain.crt \
-e REGISTRY_HTTP_TLS_KEY=/run/secrets/domain.key \
--publish published=443,target=443 \
--replicas 1 \
registry:2
You can access the service on port 443 of any swarm node. Docker sends the requests to the node which is running the service.
Load balancing considerations
One may want to use a load balancer to distribute load, terminate TLS or provide high availability. While a full load balancing setup is outside the scope of this document, there are a few considerations that can make the process smoother.
The most important aspect is that a load balanced cluster of registries must share the same resources. For the current version of the registry, this means the following must be the same:
- Storage Driver
- HTTP Secret
- Redis Cache (if configured)
Differences in any of the above cause problems serving requests. As an example, if you’re using the filesystem driver, all registry instances must have access to the same filesystem root, on the same machine. For other drivers, such as S3 or Azure, they should be accessing the same resource and share an identical configuration. The HTTP Secret coordinates uploads, so also must be the same across instances. Configuring different redis instances works (at the time of writing), but is not optimal if the instances are not shared, because more requests are directed to the backend.
Important/Required HTTP-Headers
Getting the headers correct is very important. For all responses to any
request under the “/v2/” url space, the Docker-Distribution-API-Version
header should be set to the value “registry/2.0”, even for a 4xx response.
This header allows the docker engine to quickly resolve authentication realms
and fallback to version 1 registries, if necessary. Confirming this is setup
correctly can help avoid problems with fallback.
In the same train of thought, you must make sure you are properly sending the
X-Forwarded-Proto
, X-Forwarded-For
, and Host
headers to their “client-side”
values. Failure to do so usually makes the registry issue redirects to internal
hostnames or downgrading from https to http.
A properly secured registry should return 401 when the “/v2/” endpoint is hit
without credentials. The response should include a WWW-Authenticate
challenge, providing guidance on how to authenticate, such as with basic auth
or a token service. If the load balancer has health checks, it is recommended
to configure it to consider a 401 response as healthy and any other as down.
This secures your registry by ensuring that configuration problems with
authentication don’t accidentally expose an unprotected registry. If you’re
using a less sophisticated load balancer, such as Amazon’s Elastic Load
Balancer, that doesn’t allow one to change the healthy response code, health
checks can be directed at “/”, which always returns a 200 OK
response.
Restricting access
Except for registries running on secure local networks, registries should always implement access restrictions.
Native basic auth
The simplest way to achieve access restriction is through basic authentication
(this is very similar to other web servers’ basic authentication mechanism).
This example uses native basic authentication using htpasswd
to store the
secrets.
Warning: You cannot use authentication with authentication schemes that send credentials as clear text. You must configure TLS first for authentication to work.
-
Create a password file with one entry for the user
testuser
, with passwordtestpassword
:$ mkdir auth $ docker run \ --entrypoint htpasswd \ registry:2 -Bbn testuser testpassword > auth/htpasswd
-
Stop the registry.
$ docker container stop registry
-
Start the registry with basic authentication.
$ docker run -d \ -p 5000:5000 \ --restart=always \ --name registry \ -v "$(pwd)"/auth:/auth \ -e "REGISTRY_AUTH=htpasswd" \ -e "REGISTRY_AUTH_HTPASSWD_REALM=Registry Realm" \ -e REGISTRY_AUTH_HTPASSWD_PATH=/auth/htpasswd \ -v "$(pwd)"/certs:/certs \ -e REGISTRY_HTTP_TLS_CERTIFICATE=/certs/domain.crt \ -e REGISTRY_HTTP_TLS_KEY=/certs/domain.key \ registry:2
-
Try to pull an image from the registry, or push an image to the registry. These commands fail.
-
Log in to the registry.
$ docker login myregistrydomain.com:5000
Provide the username and password from the first step.
Test that you can now pull an image from the registry or push an image to the registry.
X509 errors: X509 errors usually indicate that you are attempting to use a self-signed certificate without configuring the Docker daemon correctly. See run an insecure registry.
More advanced authentication
You may want to leverage more advanced basic auth implementations by using a proxy in front of the registry. See the recipes list.
The registry also supports delegated authentication which redirects users to a specific trusted token server. This approach is more complicated to set up, and only makes sense if you need to fully configure ACLs and need more control over the registry’s integration into your global authorization and authentication systems. Refer to the following background information and configuration information here.
This approach requires you to implement your own authentication system or leverage a third-party implementation.
Deploy your registry using a Compose file
If your registry invocation is advanced, it may be easier to use a Docker
compose file to deploy it, rather than relying on a specific docker run
invocation. Use the following example docker-compose.yml
as a template.
registry:
restart: always
image: registry:2
ports:
- 5000:5000
environment:
REGISTRY_HTTP_TLS_CERTIFICATE: /certs/domain.crt
REGISTRY_HTTP_TLS_KEY: /certs/domain.key
REGISTRY_AUTH: htpasswd
REGISTRY_AUTH_HTPASSWD_PATH: /auth/htpasswd
REGISTRY_AUTH_HTPASSWD_REALM: Registry Realm
volumes:
- /path/data:/var/lib/registry
- /path/certs:/certs
- /path/auth:/auth
Replace /path
with the directory which contains the certs/
and auth/
directories.
Start your registry by issuing the following command in the directory containing
the docker-compose.yml
file:
$ docker-compose up -d
隔離されたデータセンターでの配慮
You can run a registry in an environment with no internet connectivity. However, if you rely on any images which are not local, you need to consider the following:
-
You may need to build your local registry’s data volume on a connected host where you can run
docker pull
to get any images which are available remotely, and then migrate the registry’s data volume to the air-gapped network. -
Certain images, such as the official Microsoft Windows base images, are not distributable. This means that when you push an image based on one of these images to your private registry, the non-distributable layers are not pushed, but are always fetched from their authorized location. This is fine for internet-connected hosts, but not in an air-gapped set-up.
In Docker 17.06 and higher, you can configure the Docker daemon to allow pushing non-distributable layers to private registries, in this scenario. This is only useful in air-gapped set-ups in the presence of non-distributable images, or in extremely bandwidth-limited situations. You are responsible for ensuring that you are in compliance with the terms of use for non-distributable layers.
-
Edit the
daemon.json
file, which is located in/etc/docker/
on Linux hosts andC:\ProgramData\docker\config\daemon.json
on Windows Server. Assuming the file was previously empty, add the following contents:{ "allow-nondistributable-artifacts": ["myregistrydomain.com:5000"] }
The value is an array of registry addresses, separated by commas.
Save and exit the file.
-
Restart Docker.
-
Restart the registry if it does not start automatically.
-
When you push images to the registries in the list, their non-distributable layers are pushed to the registry.
Warning: Non-distributable artifacts typically have restrictions on how and where they can be distributed and shared. Only use this feature to push artifacts to private registries and ensure that you are in compliance with any terms that cover redistributing non-distributable artifacts.
-
次のステップ
More specific and advanced information is available in the following sections:
- Configuration reference
- Working with notifications
- Advanced “recipes”
- Registry API
- Storage driver model
- Token authentication


While there are many other sites out there for prospective car buyers, most offer little in the way of individualized content and expert advice.
Motodex enables associates to create their own curated "collections", and earn rewards from online marketing
Are you a social media influencer? Find out how you can earn rewards as a sales agent on Motodex.
We've got you covered. Using technology alongside with domain experts, Motodex can search across online classified listings and find that rare vehicle, or just find the best deal for you.


observers
Observer classes respond to life cycle callbacks to implement trigger-like behavior outside the original class. This is a great way to reduce the clutter that normally comes when the model class is burdened with functionality that doesn't pertain to the core responsibility of the class. Mongoid's observers work similar to ActiveRecord's. Example:
class ArticleObserver < Mongoid::Observer def after_save(article<EMAIL_ADDRESS>"New article", article).deliver end end
Observers are available for any document, whether it is embedded within another document or not. See callbacks for the list of available callbacks.Note that to be efficient, Mongoid only fires the observers of the document that the persistence action was executed on. This is that Mongoid aims to support large hierarchies and to handle optimized atomic updates callbacks can't be firing all over the document hierarchy.
instantiation
Observers will not be invoked unless they are instantiated first. If you are using Rails, Mongoid will instantiate your observers automatically as long as you register them in your config/application.rb file like so:
config.mongoid.observers = :article_observer, :audit_observer
If you're not using Rails, then you will have to load and register your observers directly with Mongoid and afterwards instruct Mongoid to instantiate them before they will work. Instantiating an observer registers it with its observed model(s) so they will need to be loaded beforehand.
require 'article_observer' require 'audit_observer' Mongoid.observers = ArticleObserver, AuditObserver Mongoid.instantiate_observers
mapping
Observers will by default be mapped to the class with which they share a name. So CommentObserver will be tied to observing Comment, ProductManagerObserver to ProductManager, and so on. If you want to name your observer differently than the class you're interested in observing, you can use the Observer.observe class method which takes either the concrete class (Product) or a symbol for that class (:product). If an observer needs to watch more than one kind of object, this can be specified with multiple arguments.
class AuditObserver < Mongoid::Observer observe :account, :balance def after_update(record) AuditTrail.new(record, "UPDATED") end end