DART Manhattan Release Notes
Dart Overview /
Notes for Current Users /
Non-backwards Compatible Changes /
New Features /
Supported Models /
Changed Models /
New Forward Operators /
New Observations /
New Diagnostics and Documentation /
New Utilities /
Known Problems /
Terms of Use
Dart Overview
The Data Assimilation Research Testbed (DART) is designed to
facilitate the combination of assimilation algorithms, models,
and real (or synthetic) observations to allow
increased understanding of all three.
The DART programs are highly portable, having been
compiled with many Fortran 90 compilers
and run on linux compute-servers, linux clusters, OSX laptops/desktops,
SGI Altix clusters, supercomputers running AIX, and more.
Read the
Customizations section
for help in building on new platforms.
DART employs a modular programming approach to apply an Ensemble Kalman Filter
which adjusts model values toward a state that is more consistent with information
from a set of observations. Models may be swapped in and out, as can
different algorithms in the Ensemble Kalman Filter. The method
requires running multiple instances of a model to generate an ensemble of
states. A forward operator appropriate for the type of observation being assimilated
is applied to each of the states to generate the model's estimate of the observation.
Comparing these estimates and their uncertainty to the observation and
its uncertainty ultimately results in the adjustments to the model states.
See the DART_LAB
demos or read more in the DART tutorial.
DART diagnostic output can be written that contains the model
state before and after the adjustment, along with the ensemble mean
and standard deviation, and prior or posterior inflation values if inflation is enabled.
There is also a text file, obs_seq.final, with the model estimates of the
observations. There is a suite of MATLAB® functions that facilitate exploration of the
results, but the netCDF files are inherently portable and contain all the
necessary metadata to interpret the contents with other analysis programs
such as NCL, R, etc.
To get started running with Lorenz 63 model refer to
Getting Started
Notes for Current Users
If you have been updating from the rma_trunk branch of the DART subversion
repository you will notice that the code tree has been simplified to be more
intuitive for users. The new top level directory structure looks like :
- README
- COPYRIGHT
- assimilation_code
- build_templates
- diagnostics
- documentation
- models
- observations
if you do try to do an 'svn update' on an existing directory, you will
encounter many 'tree conflicts'.
We suggest that current users checkout a fresh version of Manhattan in
a new location.
To see which files need to be moved, run 'svn status' on your original
checked out version.
Anything with an M or ? in the first column needs to be moved
to the new location in the new tree. Please contact
DART if you have any issues migrating your existing code to the new tree structure.
There is a list of non-backwards compatible changes
(see below), and a list
of new options and functions.
The Manhattan release will continue to be updated for the next few months
as we continue to add features.
Checking out the Manhattan release branch and running 'svn update'
from time to time is the recommended way to update your DART tree.
Non-backwards Compatible Changes
Unlike previous releases of DART, this version contains
more non-backwards compatible changes than usual. Please examine the
following list carefully. We do suggest you check out the Manhattan
release into a new location and migrate any local changes from previous
versions as a second step.
Changes in the Manhattan release (15 May 2015) which are not backwards compatible
with the Lanai release (13 Dec 2013):
-
We no longer require model data to be converted to DART format restart files.
We directly read and write NetCDF format only. To specify the input and output
files for filter, there are new namelist items in the &filter_nml namelist:
'input_state_file_list'
and 'output_state_file_list'
.
-
The information formerly in
Prior_Diag.nc and Posterior_Diag.nc
has been moved. If you are reading and writing ensemble members from
different files, the state information, the ensemble mean and standard deviation,
and the inflation mean and standard deviation will all be read and written to
separate files:
- [stage]_member_####.nc
- [stage]_mean.nc
- [stage]_sd.nc
- [stage]_priorinf_{mean,sd}.nc (if prior inflation is turned on)
- [stage]_postinf_{mean,sd}.nc (if posterior inflation is turned on)
If you are reading and writing ensemble members from a
single file, all this information will now be in a single NetCDF file but
will be stored in different variables inside that file:
- [var].nc
- [var]_mean.nc
- [var]_sd.nc
- [var]_priorinf_{mean,sd}.nc (if prior inflation is turned on)
- [var]_postinf_{mean,sd}.nc (if posterior inflation is turned on)
We also now have options for writing files at four stages of the
assimilation cycle:
'input', 'preassim', 'postassim', 'output'
.
This is set in the &filter_nml namelist with stages_to_write.
- New model_mod.f90 required routines:
- vert_convert()
- query_vert_localization_coord()
- pert_model_copies()
- read_model_time()
- write_model_time()
There are default version of these available to use if you have
no special requirements.
- Several of the model_mod.f90 argument lists have changed
- model_interpolate() now takes in
the
state_handle
as an argument rather than
a state vector array. It also return an array of
expected_obs
and istatus
for
each of the ensemble members
- get_state_meta_data() also requires
the
state_handle
as an argument rather than
a state vector array.
- nc_write_model_atts() has an additional
argument
moel_mod_writes_state_variables
. If
true then the model_mod is expected to write out the state variables,
if false DART will write out the state variable (this is the prefered
method for adding new models, it requires less code from the model
developer)
-
There are several namelist changes mainly in the &filter_nml and
&perfect_model_mod which are outlined in detail in
Manhattan_diffs_from_Lanai
-
All modules have been moved to
DART/assimilation_code/modules/
directory. And similarly all of the programs have moved to
DART/assimilation_code/programs/
-
The location modules which were stored in locations
have moved to DART/assimilation_code/location
directory
-
The observation converters which were stored in observations
have moved to DART/observations/obs_converters
directory
-
The forward operators have moved from obs_def/obs_def_*_mod.f90 to
observations/forward_operators
-
The tutorial files have moved to
DART/docs/tutorial directory
-
The program fill_inflation_restart is OBSOLETE since
DART inflation files are now in NetCDF format. Now inflation files can be filled
using ncap2. Here is an example using version 4.4.2 or later
of the NCO tools:
ncap2 -s "T=1.0;U=1.0;V=1.0" wrfinput_d01 prior_inf.nc'
ncap2 -s "T=0.6;U=0.6;V=0.6" wrfinput_d01 prior_sd.nc'
-
The default flags in the mkmf_template.XXX files have been updated to
be more consistent with current compiler versions.
- If you enable the sampling error correction option, the required
data is now read from a single netcdf file which supports multiple
ensemble sizes. A program is provided to compute additional ensemble
sizes if they are not in the default file.
-
Our use of TYPES and KINDS has been very confusing in
the past. In Manhattan we have tried to make it clearer
which things in DART are generic quantities (QTY) - temperature,
pressure, etc - and which things are specific types of
observations - Radiosonde_temperature, Argo_salinity etc.
Below is a mapping between old and new subroutine names
here for reference. We have made these changes to all files
distributed with DART.
If you have lots of code developed outside of the subversion
repository, please contact
DART for a sed script to
help automate the changes.
Public subroutines, existing name on left, replacement on right:
assimilate_this_obs_kind() => assimilate_this_type_of_obs(type_index)
evaluate_this_obs_kind() => evaluate_this_type_of_obs(type_index)
use_ext_prior_this_obs_kind() => use_ext_prior_this_type_of_obs(type_index)
get_num_obs_kinds() => get_num_types_of_obs()
get_num_raw_obs_kinds() => get_num_quantities()
get_obs_kind_index() => get_index_for_type_of_obs(type_name)
get_obs_kind_name() => get_name_for_type_of_obs(type_index)
get_raw_obs_kind_index() => get_index_for_quantity(qty_name)
get_raw_obs_kind_name() => get_name_for_quantity(qty_index)
get_obs_kind_var_type() => get_quantity_for_type_of_obs(type_index)
get_obs_kind() => get_obs_def_type_of_obs(obs_def)
set_obs_def_kind() => set_obs_def_type_of_obs(obs_def)
get_kind_from_menu() => get_type_of_obs_from_menu()
read_obs_kind() => read_type_of_obs_table(file_unit, file_format)
write_obs_kind() => write_type_of_obs_table(file_unit, file_format)
maps obs_seq nums to specific type nums, only used in read_obs_seq:
map_def_index() => map_type_of_obs_table()
removed this. apparently unused, and simply calls get_obs_kind_name():
get_obs_name()
apparently unused anywhere, removed:
add_wind_names()
do_obs_form_pair()
Public integer parameter constants and subroutine formal argument names,
old on left, new on right:
KIND_ => QTY_
kind => quantity
TYPE_ => TYPE_
type => type_of_obs
integer parameters:
max_obs_generic => max_defined_quantities (not currently public, stays private)
max_obs_kinds => max_defined_types_of_obs
-
For smaller models we support single file input and output. These files contain all of
the member information, mean, standard deviation and inflation values for all of the
state variables. This can be run with cycling and all time steps will be appended
to the file.
For perfect_model_obs we provide a
perfect_input.cdl file which contains a single ensemble member
which will be considered the 'truth' and observations will be generated based on
those values. The output will contain all of the cycling timesteps all of the state
variables.
For filter we provide a
filter_input.cdl file which contains all of the state member
variables and potentially inflation mean and standard deviation values. The
output will contain all of the cycling timesteps all of the state variables.
Additionally you have the option to write out different stages during the
assimilation in the &filter_nml stages_to_write
mentioned above.
To generate a NetCDF file from a .cdl file run:
ncgen -o perfect_input.nc perfect_input.cdl
ncgen -o filter_input.nc filter_input.cdl
New Features
- DART now reads and writes NetCDF files for the model state information.
If your model uses NetCDF file format, you no longer need model_to_dart or
dart_to_model to translate to a DART format file. If your model does not
use NetCDF, you can adapt your model_to_dart and dart_to_model executables
to read and write a NetCDF file for DART to use.
The read/write code is part of the core DART routines so no code is needed
in the model_mod model-specific module. There is a new routine
add_domain()
that a model_mod::static_init_model() can user to define which NetCDF variables
should be part of the model state, and what DART quantity (formerly kind) they
correspond to.
- DART no longer limits the size of a model state to the size of a single
MPI task's memory. The state is read in variable by variable and distributed
across all MPI tasks, so the memory use is much smaller than previous versions
of DART. One-sided MPI communication is used during the computation of
forward operator values to get required parts of the state from other tasks.
- Many of the DART namelists have been simplified, and some items have
moved to a more specific namelist.
- Observation sequence files can include externally computed forward operator
values which can be used in the assimilation instead of calling a forward
operator inside DART.
- The DART directory structure has been reorganized to make it easier to
identify the various software tools, modules, documentation and tutorials
supplied with the system.
- The MATLAB® diagnostic routines have been updated to not require the
MEXNC toolbox. These routines use the built-in NetCDF support that comes
with MATLAB®.
- There is a new Particle Filter type. Please contact us if you
are interested in using it.
- DART can now take subsets of observation types and restrict them
from impacting certain quantities in the state during the assimilation.
A tool to simplify constructing the table of interactions is provided
(obs_impact_tool).
- State Structure
- Contains information about dimensions and size of variables in your
state. There is a number of accessor functions to get variable information
such as
get_variable_size()
.
See the state_structure.html
for more details.
- The POP model_mod now can interpolate Sea Surface Anomaly observations.
Supported Models
Currently we support the models listed below. There are several new
models that have been added that are not on the Lanai Release
including CM1, CICE, and ROMS. Any previously supported models not on
this list are still supported in DART classic
- 9var
- DART interface documentation for the
9var model.
- bgrid_solo
- DART interface documentation for the
bgrid solo model.
- cam-fv
- cice (NEW)
- DART interface documentation for the
CICE model.
- Documentation for the
CICE model.
- cm1 (NEW)
- forced_lorenz_96
- lorenz_63
- DART interface documentation for the
lorenz_96 model.
- lorenz_84
- DART interface documentation for the
lorenz_84 model.
- lorenz_96
- DART interface documentation for the
lorenz_96 model.
- lorenz_04
- DART interface documentation for the
lorenz_04 model.
- mpas_atm (NetCDF overwrite not supported for update_u_from_reconstruct = .true. )
- POP
- DART interface documentation for the
POP global ocean model.
- Documentation for the
POP model.
- ROMS (NEW)
- DART interface documentation for the
ROMS regional ocean model.
- Documentation for the
ROMS model.
- simple_advection
- wrf
- DART interface documentation for the
WRF regional forecast model.
- Documentation for the
WRF model.
The DART/models/template directory contains sample
files for adding a new model. See the
Adding a Model section of the DART web pages for more help on adding a new model.
Changed Models
- WRF
- Allow advanced microphysics schemes (needed interpolation for 7 new kinds)
- Interpolation in the vertical is now done in log(p) instead of linear
pressure space. log(p) is the default, but a compile-time variable can
restore the linear interpolation.
- Added support in the namelist to avoid writing updated fields back into
the wrf netcdf files. The fields are still updated during the assimilation
but the updated data is not written back to the wrfinput file during the
dart_to_wrf step.
- Fixed an obscure bug in the vertical convert routine of the wrf model_mod
that would occasionally fail to convert an obs. This would make tiny
differences in the output as the number of mpi tasks change. No quantitative
differences in the results but they were not bitwise compatible before and they
are again now.
- CAM
- DART/CAM now runs under the CESM framework, so all options available
with the framework can be used.
- Support for the SE core (HOMME) has been developed but is NOT part
of this release. Please contact the DART
Development Group if you have an interest in this configuration of CAM.
- Simple Advection Model
- Fixed a bug where the random number generator was being
used before being called with an initial seed.
New Observation Types/Forward Operators
- Many new observation types related to land and atmospheric chemistry
have been added. See the obs_kind_mod.f90 for
a list of the generic quantities now available.
- New forward operator for Sea Ice (cice) ice thickness observations. See the
obs_def_cice_mod.f90 file for details.
- New forward operator for Carbon Monoxide (CO) Nadir observations. See the
obs_def_CO_Nadir_mod.f90 file for details.
- New forward operator for Total Cloud Water in a column observations. See the
obs_def_cwp_mod.f90 file for details.
New Observation Types/Sources
- AVISO
Added an observation converter for Sea Surface Height Anomaly observations.
Documentation in
convert_aviso.f90 (source).
- cice
Added an obs_sequence converter for Sea Ice observations.
Documentation in
cice_to_obs.html.
- GPSPW
Added an obs_sequence converter for GPS precipitable water observations.
Documentation in
convert_gpspw.f90 (source).
- MODIS
Added an obs_sequence converter for MODIS FPAR (Fraction of Photosynthetically Active Radiation)
and LAI (Leaf Area Index) obseverations.
Documentation in
MOD15A2_to_obs.html.
- ok_mesonet
Added an obs_sequence converter for the Oklahoma Mesonet observations.
Documentation in
ok_mesonet.html.
- ROMS
Added an obs_sequence converter for ROMS ocean data. This converter includes externally computed
forward operators output from the ROMS model using FGAT (First Guess At Time) during the model run.
Documentation in
convert_roms_obs.f90 (source).
- SSUSI
Added an obs_sequence converter for wind profiler observations.
Documentation in
convert_f16_edr_dsk.html.
- tropical_cyclone
Added an obs_sequence converter for ASCII format tropical cyclone track observations.
Documentation in
tc_to_obs.html.
New Diagnostics and Documentation
Better Web Pages.
We've put a lot of effort into expanding our documentation.
For example, please check out
the MATLAB diagnostics section or the pages outlining the
observation sequence file contents.
- The MATLAB® diagnostic routines have been updated to remove the dependency on third-party
toolboxes. These routines use the built-in netCDF support that comes
with basic MATLAB® (no other toolboxes needed).
But there's always more to add.
Please let us know where we are lacking.
New Utilities
This section describes
updates and changes to the tutorial materials,
scripting, setup, and build information
since the Lanai release.
- obs_impact_tool please refer to
Website
or local file
- gen_sampling_error_table now computes sampling error correction tables for any ensemble size.
- compute_error
Website
or local file
Known Problems
-
There are many changes in this release and more updates are
expected to come soon. We are not aware of any obvious bugs, but
if you encounter any unexpected behavior please contact us.
Please watch the dart-users email list for announcements of updates
to the release code, and be prepared to do an 'svn update' from time
to time to get updated files.
Terms of Use
DART software - Copyright UCAR. This open source software is provided
by UCAR, "as is", without charge, subject to all terms of use at
http://www.image.ucar.edu/DAReS/DART/DART_download


TOPICAL HEMO-STAT- aluminum chloride solution
Henry Schein
Disclaimer: This drug has not been found by FDA to be safe and effective, and this labeling has not been approved by FDA. For further information about unapproved drugs, click here.
Soak a cotton-tipped applicator or a pledget of cotton with Hemo-Stat. Apply to bleeding area with pressure until hemorrhage is arrested.
HENRY SCHEIN®
TOPICAL HEMO-STAT SOLUTION
REF 100-2645
Contains: 25% Aluminum Chloride in an aqueous vehicle.
For Professional Use Only
Warning: Corrosive Consult MSDS
Distributed by:
HENRY SCHEIN INC.
Melville, NY 11747 USA
Henry Schein U.K. Holdings Ltd.
Gillingham ME 8 0SB U.K.
1 FL. OZ. (30 ml)
MADE IN USA
|
TOPICAL HEMO-STAT
aluminum chloride solution | ||||||||||||||||||||
| ||||||||||||||||||||
| ||||||||||||||||||||
| ||||||||||||||||||||
| ||||||||||||||||||||
|
| Labeler - Henry Schein (012430880) |
| Registrant - DSHealthcare (056296981) |
| Establishment | |||
| Name | Address | ID/FEI | Business Operations |
|---|---|---|---|
| DENTSPLY Caulk | 083235549 | MANUFACTURE |


Splitting Made Easy
With Split-the-Bill's "Even Split" option you simply enter the total, number of people, and hit the split button.
With Split-the-Bill's "Even Split" option you simply enter the total, number of people, and hit the split button.
Fully customizable amounts. Set the amount to be split for each person to whatever you like down to the penny.
Keep track of who has paid and who hasn't paid. Get a quick view of this in the dashboard. More details are available by clicking into one of your splits.


# Data will be destroyed after you
# access it, or after 24 hours.
# Enter URL below to send
# base64 encoded files.
# ⚠768kb limit!
# <== Store JSON
# Write a valid JSON string
# and you'll get JSON.
{
"account": {
"username": "foobar",
"password": "secret"
}
}
# <== Store Strings
# Write anything else
# and you'll get a string.
Be excellent to each other! – Bill


Additional supporting information may be found in the online version of this article at the publisher's website.
Figure S1 Red‐backed voles ( Myodes rutilus) instantaneous rate of change from autumn to the following spring in relation to winter maximum snow depth, 2000–2018 at Kluane Lake. There is an indication of a weak positive relationship ( R 2 = 0.15, P = 0.10. Average snow depth over winter is highly correlated with maximum snow depth ( r = 0.82, n = 18).
Figure S2 Deer mouse ( Peromyscus maniculatus) autumn population density 1976–2018 at Kluane Lake, with 95% confidence limits. No deer mice were live trapped in the forest between 1990 and 1995 in spite of extensive trapping. These data are discussed in Krebs et al. (2018b). Figure updated from Krebs et al. (2018b) with permission.


Goodbye, 20th Century! (1999) Reviewed by Eugene Novikov http://www.ultimate-movie.com/ Member: Online Film Critics Society
"The future is as screwed up as the past."
Starring Lazar Ristovski. Not Rated (the equivalent of a strong R, for gory violence and an explicit sex scene).
Goodbye, 20th Century is one of only six films produced in the Republic of Macedonia. It's the only one of the six that I have seen and it hasn't made me especially eager to see more. It's a frenetic, nonsensical doo-hickey of a science fiction movie that makes no effort to be at all coherent and, as such, is a big mess. It's utterly pointless -- or maybe it's just me. Either way, I'm at a loss.
The film is composed of three vignettes. The first plays like a pornographic Tuck Everlasting.  Or something. The second is brief and is "set" at the end of the 19th century. This one might have something to do with someone marrying his own sister. Or something. The last one concerns a deranged Santa Claus on New Year's Eve 1999 and ends in a bloodbath and the words "Goodbye, 20th Century!"
Or something. I don't know. Maybe it would have all become clearer on repeat viewing, but I wasn't going to go through that. The fact of the matter is, to a one-time viewer not familiar with Macedonian cinema, Goodbye, 20th Century makes absolutely no sense. Lines and characters seem to come out of nowhere, without any apparent thought behind them. The running theme of incest is at best curious and at worst repulsive; either way, why it's there remains a mystery. The film also tries to interweave the three stories, but the result is even more disjointed. There may well be some ideas brimming underneath the surface here, in which case they simply never see the light of day.
>From a purely technical perspective, this is a reasonably well-made movie. Action scenes are filmed with an effectively surrealistic style and there is enough deft camerawork to justify the filmmakers being in the business. The sets are gorgeously bleak and they don't reek of a low budget. I also like the way the film's finale was staged and filmed, though I didn't really understand that either. Goodbye, 20th Century has enough visual panache that, if coupled with a good plot, which it is laughing, it would be a fair contender for the Best Foreign Film Oscar this year.
I can't help but think that there was something profound that directors Darko Mitrevski and Aleksandar Popovski were trying to tell us here. If only I knew what it was. Before I can perceive understated messages I have to at least get a jist of the main story line and with this film I wasn't even fully successful there. In fact, nobody can be expected to understand the theme of a movie before they figure out what it is about. That should be the #1 piece of advice at film schools everywhere.
There is no way I can completely dismiss this film considering its technical proficiency and the nagging possibility that, um, I didn't get it. But my argument stands: to most American filmgoers, Goodbye, 20th Century will remain a mystery. The story never came together sufficiently to work on any level other than the visual.
Grade: C-
©1999 Eugene Novikov
Sent via Deja.com http://www.deja.com/ Before you buy.
The review above was posted to the
rec.arts.movies.reviews newsgroup (de.rec.film.kritiken for German reviews).
The Internet Movie Database accepts no responsibility for the contents of the
review and has no editorial control. Unless stated otherwise, the copyright
belongs to the author.
Please direct comments/criticisms of the review to relevant newsgroups.
Broken URLs inthe reviews are the responsibility of the author.
The formatting of the review is likely to differ from the original due
to ASCII to HTML conversion.
Related links: index of all rec.arts.movies.reviews reviews


App Engine Python SDK
v1.6.9 rev.445
The Python runtime is available as an experimental Preview feature.
google
appengine
api
modules
modules_service_pb
GetVersionsRequest
Public Member Functions
|
Public Attributes
|
Static Public Attributes
|
List of all members
google.appengine.api.modules.modules_service_pb.GetVersionsRequest Class Reference
Inheritance diagram for google.appengine.api.modules.modules_service_pb.GetVersionsRequest:
Public Member Functions
def
__init__
def
module
def
set_module
def
clear_module
def
has_module
def
MergeFrom
def
Equals
def
IsInitialized
def
ByteSize
def
ByteSizePartial
def
Clear
def
OutputUnchecked
def
OutputPartial
def
TryMerge
def
__str__
Public Attributes
has_module_
module_
Static Public Attributes
int
has_module_
= 0
string
module_
= ""
int
kmodule
= 1
The documentation for this class was generated from the following file:
code/googleappengine-read-only/python/google/appengine/api/modules/modules_service_pb.py
Maintained by
tzmartin


struct chessboard_t {
bool ok;
std::string config_file;
int nb_rows;
int nb_cols;
double square_size;
std::vector<cv::Point3f> object_points;
chessboard_t();
chessboard_t(const std::string &config_file_);
~chessboard_t();
};
Chessboard
int chessboard_load(chessboard_t &cb, const std::string &config_file);
Load config file
Returns 0 for success, -1 for failure
std::vector<cv::Point3f> chessboard_object_points(const chessboard_t &cb);
Create object points
Returns Vector of object points
int chessboard_detect(const chessboard_t &cb,
const cv::Mat &image,
std::vector<cv::Point2f> &corners);
Detect chessboard corners
Returns 0 for success, -1 for failure
int chessboard_draw_corners(const chessboard_t &cb, cv::Mat &image);
Draw chessboard corners
Returns 0 for success, -1 for failure
int chessboard_solvepnp(const chessboard_t &cb,
const std::vector<cv::Point2f> corners,
const mat3_t &K,
mat4_t &T_CF,
matx_t &X);
Solve PnP between camera and chessboard
Returns 0 for success, -1 for failure
void chessboard_project_points(const chessboard_t &cb,
const matx_t &X,
const mat3_t &K,
cv::Mat &image);
Project 3D points to image. Where the corner positoins relative to the
camera is in ideal coordinates, and K
is the camera matrix.


---
layout: minimal
title: "About me"
permalink: /about/index.html
description: "Curious soul 24x7. Software engineer by day. Reading/couch potating by night."
---
TL;DR life story
Grew up in Faridabad, a suburban town near Delhi, India.
Childhood spent nerding out on Pokemon and Harry Potter, fiddling with computers, playing FIFA, and headbanging to Linkin Park.
Went to undergrad school at NSIT, University of Delhi to study computer engineering.
Currently finishing my Masters degree in Computer Science at Stony Brook University, New York.
Hobbies
- Reading. Used to read a lot of fiction earlier. Slowly learning to assimilate non-fiction
- Football/soccer. Passionate Arsenal fan since 2006
- Hiking/nature meandering. New to this, started enjoying this recently
- Music. Don't have a favorite genre or band, different tunes for different moods. I DJ occasionally for my friends.
- Netflix. Hate to admit it, but waste a lot of time on there
Professional Skills
Here's my software engineering skill-set.
Things I'm good at
- System design
- Backend Engineering(>4 years of Java, >2 for Python and NodeJS)
- Distributed systems concepts
- Working with misc datastores: MongoDB, MySQL, Elasticsearch, Redis etc
- Using cloud platforms and tools(Kubernetes, Openstack, GCP, AWS)
- Android development(don't do this actively)
Things I'm fairly comfortable with
- Infrastructure management(Terraform, Ansible)
- Bash and Python scripting
- Web development(Angular, ReactJS) (don't do this actively)


'The King Has Tooted'. NBA Superstar LeBron James Farts at Fans Who Asked For Autograph
The athlete described by many as the greatest basketball player of all time and often compared to another basketball legend Michael Jordan, has recently angered fans by commenting on protests in Hong Kong.
NBA superstar LeBron James may find himself in hot water after a video of him farting at fans, who asked for an autograph, went viral. The footage shows the Lakers forward leaving the Mahogany Prime Steakhouse in Oklahoma City with another basketball player Chris Paul and approaching a black SUV. Two fans hoping to get an autograph wished the athlete good luck in the upcoming came, but the superstar simply lifted his leg and broke wind, which was so loud that it was recorded by the camera’s audio from the other side of the parking lot.
The fans apparently were ok with his move and started laughing, but netizens didn’t like the basketball superstar's behaviour. Not one bit.
Some users said that LeBron James doesn’t deserve to be compared to another basketball GOAT – Greatest Of All time – Michael Jordan, because the latter always behaved like a gentleman
Some users even said that athletes’ disrespectful behaviour was the reason why they stopped watching sports.
Other users questioned how this story got in the news.
A few of LeBron's fans weren't even bothered by his behaviour. Some even liked it…


beach hut
Hire Us
Services
Free Consultation
Menu
If you need help planning a web project or have one ready to go why not reach out. We can help.
tim on 01 5045638
<EMAIL_ADDRESS>About us
The team
Our services
Our projects
Work with us
Start a project
Contact Us
Blog
Follow us
Facebook
Instagram
Twitter
Option_1
contact us
01 504 5638
<EMAIL_ADDRESS>Option_2
Let Us Contact You
Name
Company
Email
Project description
126 sarsfield Park, Lucan, Dublin
Top


We employ simulations with the National Center for Atmospheric Research’s Community Earth System Model (CESM) version 1.0.4 30, which incorporates the Los Alamos Sea Ice Model version 4 (CICE4 49), the Community Land Model version 4 (CLM4 50), and the Community Atmosphere Model version 4 (CAM4 51), coupled to a slab ocean. The atmosphere and land components are simulated on a 0.9° × 1.5° grid, whereas the ice and ocean components are on an approximately 1° × 1° grid. A total of 12 simulations are conducted under preindustrial atmospheric CO 2 concentrations (284.7 ppm) with various modifications to the ocean surface or mixed layer heat flux (Q-flux) and land topography (Table 1).
We use the routine available in the standard CESM download, which uses monthly oceanic fields of temperature, salinity, surface velocities, boundary layer depth, surface heat flux, melting and freezing fluxes to generate the Q-flux for a specific mixed layer depth. The mixed layer is an annual average to balance the seasonal cycle 52. We use an 80-year average from a multi-millennia coupled control simulation with preindustrial forcing to generate a climatological reference state, which globally and annually average to a zero Q-flux. The annual net Q-flux is a small residual of a strong seasonal cycle, with the same pattern as the surface heat flux, only with a smaller amplitude. In the equilibrium state, the heat flux at the surface and mixed layer depth are the same, since the slab-ocean simulation does not allow lateral transport. The annual-mean climatological Q-flux compares well with observations and is dominated by tropical upwelling regions taking up heat and Western boundary currents, the Nordic seas, and the Southern Ocean releasing heat to the atmosphere 31, 53.
We then apply the globally non-zero Q-flux pattern as a negative (i.e., heat flux out of the mixed layer depth into the ocean) and positive (i.e., heat flux into atmosphere) ocean heat uptake forcing inducing a global warming and cooling. The idealized Q-flux anomaly patterns are sine-shaped functions being bound to zero at 30°N and 60°N and rising to a maximum value at 45°N. The global imbalance was tuned to +1 W/m 2 and −1 W/m 2 global-equivalent, which implies the Pacific local maximum is smaller (31.5 W/m 2) than the Atlantic local maximum (54 W/m 2), since the Pacific area is larger (Supplementary Figure 1). We also configured a Q-flux anomaly for the North Pacific that was the equivalent area as the heat flux imposed on the North Atlantic to eliminate the potential effects of the different areas of the North Pacific and North Atlantic heat flux anomalies on global and Arctic climate. The local anomalous Q-flux is in the same order of magnitude as the climatological annual-mean flux, but smaller than the local seasonal cycle. The mixed layer depth is kept at the climatological control simulation annual average (spatially varying).
The naming convention for the simulations is as follows: NP-Warm (North Pacific +1 W/m 2), NP-Cool (North Pacific −1 W/m 2), NA-Warm (North Atlantic +1 W/m 2), NA-Cool (North Atlantic −1 W/m 2) for the single-basin Q-flux perturbations, and NP-Warm/NA-Cool and NP-Cool/NA-Warm denoting the compensating “seesaw” Q-flux perturbations. Simulations were also run with the positive ocean-to-atmosphere Q-flux perturbations in each basin with modified land topography, such that the global land surface was set to a geopotential height of zero (NP-Warm-Notopo and NA-Warm-Notopo). A simulation was run with a positive heat flux imposed in the Northeast Pacific that was approximately equivalent to the area of the Atlantic heat flux (NEP-Warm).


--- layout: post title: Builder design pattern tagline: " wrapped with Python3" permalink: /python3/design/patterns/creational/builder.html categories: [literate programming, emacs, Python3] tags: [OOP, design, patterns, creational, builder] ---
Hope this page might elucidate this pattern in a bit explicit manner.
The builder pattern is an object creation software design pattern. Unlike the abstract factory pattern and the factory method pattern whose intention is to enable polymorphism, the intention of the builder pattern is to find a solution to the telescoping constructor anti-pattern. The telescoping constructor anti-pattern occurs when the increase of object constructor parameter combination leads to an exponential list of constructors. Instead of using numerous constructors, the builder pattern uses another object, a builder, that receives each initialization parameter step by step and then returns the resulting constructed object at once.
Separate the construction of a complex object from its representation so that the same construction process can create different representations.
Other names for the pattern.
The main premise for design is an excessive number of constructors.
Builder
often builds a Composite
. Often, designs start out
using Factory Method
(less complicated, more customizable,
subclasses proliferate) and evolve toward Abstract Factory
,
Prototype
, or Builder
(more flexible, more complex) as the
designer discovers where more flexibility is needed. Sometimes
creational patterns are complementary: Builder
can use one of the
other patterns to implement which components are built.
A graphical representation of the pattern. Class diagrams and Interaction diagrams may be used for this purpose.
specifies an abstract interface for creating parts of a Product object.
constructs an object using the Builder interface.
A description of an implementation of the pattern; the solution part of the pattern.
A description of the results, side effects, and trade offs caused by using the pattern.
Examples of real usages of the pattern.
Other patterns that have some relationship with the pattern; discussion of the differences between the pattern and similar patterns.
For educational purposes it would be priceless to look at the working code sample. Exactly to achieve a stunning clarity it might be reasonable to compose a test suite for testing the pattern features and dispose it nearby the pattern source code.
Composes some particular model on demand.
| Pattern code | Test suite |
|---|---|
#!/usr/bin/env python # -*- coding: utf-8 -*- """ A SkyLark builder """ class Car(object): """Product""" def __init__(self): self.model = None self.tires = None self.engine = None def __str__(self): return '{} | {} | {}'.format( self.model, self.tires, self.engine) class Director(object): """Director""" def __init__(self, builder): self._builder = builder def construct_car(self): pass def get_car(self): pass class Builder(object): """Abstract Builder""" def __init__(self): self.car = None def create_new_car(self): self.car = Car() class SkyLarkBuilder(Builder): """Concrete Builder --> provides parts and tools to work on the parts""" def add_model(self): self.car.model = "Skylark" def add_tires(self): self.car.tires = "Regular tires" |
#!/usr/bin/env python # -*- coding: utf-8 -*- """ An attempt to illustrate how it works """ import unittest class TestSimpleBehavior(unittest.TestCase): def setUp(self): pass def tearDown(self): pass |
| Pattern code | Test suite |
|---|---|
#!/usr/bin/env python # -*- coding: utf-8 -*- """ A dwelling builder """ class Director(object): def __init__(self): self.builder = None def construct_building(self): self.builder.new_building() self.builder.build_floor() self.builder.build_size() def get_building(self): return self.builder.building class Builder(object): """ Abstract Builder """ def __init__(self): self.building = None def new_building(self): self.building = Building() def build_floor(self): raise NotImplementedError def build_size(self): raise NotImplementedError class BuilderHouse(Builder): """Concrete Builder""" def build_floor(self): self.building.floor = 'One' def build_size(self): self.building.size = 'Big' class BuilderFlat(Builder): def build_floor(self): self.building.floor = 'More than One' def build_size(self): self.building.size = 'Small' class Building(object): """Product""" def __init__(self): self.floor = None self.size = None def __repr__(self): return 'Floor: {0.floor} | Size: {0.size}'.format(self) director = Director() director.builder = BuilderHouse() director.construct_building() building = director.get_building() print(building) director.builder = BuilderFlat() director.construct_building() building = director.get_building() print(building) Floor: One | Size: Big Floor: More than One | Size: Small |
#!/usr/bin/env python # -*- coding: utf-8 -*- """ An attempt to illustrate how it works """ import unittest class TestSimpleBehavior(unittest.TestCase): def setUp(self): pass def tearDown(self): pass |


Chapter 12: Hello remix¶
Now here’s where things get fun. Your entire analysis is scripted top to bottom, which means it can be rerun and reproduced. It also be remixed.
Remember this line earlier?
my_prop = 'PROPOSITION 064- MARIJUANA LEGALIZATION. INITIATIVE STATUTE.'
That’s where we set which proposition we wanted to filter on. It was a key fork in the road, which shaped all the analysis that followed.
That means that if we substituted a different proposition name from the value_counts list just above it we could rerun our notebook and conduct an identical analysis of another proposition, without writing another line of code.
Let’s try it. I picked the death penalty ban that was on the same ballot and changed that cell of code to this:
my_prop = 'PROPOSITION 062- DEATH PENALTY. INITIATIVE STATUTE.'
Now I go to the Run menu at the top of the notebook and selected “Run All Cells.” Wait a few seconds and, boom, you’ll have a whole new of donors plotted out.


{% extends "layout.html" %} {% block meta %} {{ super() }} {% endblock %} {% block content %}
This page documents the methodology used for extracting metadata from the cable TV news video dataset. Additional detail on our methodology can be found in our technical reports; see our about us or paper page.
The Stanford Cable TV News Analyzer accesses video data provided by the Internet Archive's TV News Archive. The Internet Archive provides data on a 24 hour delay. As a result of this delay, plus the latency of additional data processing, new results appear on the Stanford Cable TV News Analyzer approximately 24-36 hours after a program's original air time.
We detected faces in video frames using the MTCNN [1,2] face detector. Due to the high cost of performing face detection on all frames in the dataset, we performed face detection on a subset of frames. Data prior to Jan. 1, 2019 is uniformly sampled every three seconds in a video. Data after Jan. 1, 2019 is sampled uniformly sampled at one frame per second. This process yielded a total of 306 million face detections (from Jan. 2010 to Jul. 2019). The Stanford Cable TV News Analyzer tabulates face screen time at the granularity of this sampling. For example, a face detection of Anderson Cooper in a single video frame (after Jan. 1, 2019) contributes one second to the estimate of Mr. Cooper's screen time in the video.
The location of each detected face is represented by an axis-aligned bounding box. To remind users that face detection is not occurring on all frames, the Stanford Cable TV News Analyzer's video player renders face bounding boxes in the frame in which a detection occurred, then fades out the box until the next sampled frame.
Validation: To estimate the precision and recall of the face detector, we manually counted the number of faces present in 2,500 randomly selected frames of the dataset (250 frames from each year from 2010-2019). We find that precision is 98.5% and precision is 74.5%. Recall is lower because the "ground truth" human annotations include all faces in the frame, including difficult to detect faces (e.g., out-of-focus, partially occluded, very small faces). A large fraction of recall errors are in frames with crowds (such as a political rally) where background faces are small and often partially occluded.
For each detected face, we compute the following per-face “tags” and descriptors:
We compute the 128-element FaceNet descriptor [3,4] from the pixels contained inside a face’s bounding box.
We tag all faces with an estimate of their presented binary gender (male/female) using a binary classifier that operates on a face's FaceNet descriptor as input. We note that treating an individual’s gender as a binary quantity, as well as assessing gender solely from an individual's appearance, is a grossly simplified treatment of a complex topic. However, we believe that binary classification can still provide useful insights into the presentation of cable TV news. (Further discussion about our decision to include gender tags is available on our FAQ.)
To train the classifier, we manually annotated the presented binary gender of 12,669 faces selected at random from the dataset. Each face was annotated by a single human annotator. (No additional consensus protocol was used.) We used these ground-truth annotations to train a binary classifier based on the precomputed FaceNet descriptors. Our implementation uses a k-NN classifier with classification results determined by majority vote with k=7.
Validation: We computed the classifier's agreement with a single human annotator on a test set of 6,000 faces. Faces were sampled from the first ten years of the dataset (Jan. 1, 2010 to Dec. 31, 2019). We stratified samples by year and channel, selecting 200 faces at random from each of the 30 slices (10 years x 3 channels). The human annotator judged 4,109 of the 6,000 test set faces (68.5%) as male-presenting individuals and 1,891 (31.5%) as female-presenting individuals.
The confusion matrix for our gender estimation model is given below. Overall the model agrees with human annotations 97.2% of the time. When a face is judged to be male-presenting by a human, the model agrees with the human 98.8% of the time. When a face is judged to be female-presenting, the model agrees with the human 93.8% of the time.
The table below gives the accuracy of our presenting-gender estimation model for each (year, channel) slice. Accuracy is judged as agreement with results provided by a human annotator.
| CNN | Fox News | MSNBC | ||||||||||
| Year | Accuracy (overall/male/female) | Accuracy (overall/male/female) | Accuracy (overall/male/female) | |||||||||
| 2010 | 97.0 | 97.9 | 95.0 | 98.5 | 100.0 | 95.1 | 99.0 | 100.0 | 96.9 | |||
| 2011 | 97.0 | 99.3 | 92.2 | 98.5 | 99.3 | 96.6 | 98.0 | 100.0 | 91.5 | |||
| 2012 | 96.5 | 99.2 | 91.3 | 95.5 | 97.1 | 92.2 | 97.5 | 97.3 | 98.1 | |||
| 2013 | 97.5 | 98.5 | 95.2 | 96.5 | 98.5 | 92.9 | 97.0 | 98.6 | 93.3 | |||
| 2014 | 98.0 | 98.3 | 97.5 | 98.0 | 100.0 | 93.7 | 99.0 | 100.0 | 97.5 | |||
| 2015 | 96.5 | 98.5 | 92.5 | 98.0 | 100.0 | 93.7 | 95.0 | 96.6 | 90.4 | |||
| 2016 | 95.0 | 98.4 | 89.6 | 96.5 | 98.6 | 91.7 | 96.5 | 97.9 | 93.1 | |||
| 2017 | 97.5 | 98.5 | 95.4 | 98.0 | 97.4 | 100.0 | 98.0 | 100.0 | 93.4 | |||
| 2018 | 98.5 | 99.3 | 96.6 | 98.0 | 99.3 | 95.1 | 95.5 | 97.0 | 92.5 | |||
| 2019 | 96.0 | 100.0 | 88.7 | 96.5 | 99.2 | 91.0 | 96.5 | 98.6 | 91.9 |
We use the Amazon Rekognition Celebrity Recognition API to identify the detected faces. The API provides a face identity estimate only when its identification confidence score is greater than 0.5. We use all predictions above this 0.5 threshold, and do no additional thresholding. As a result of this process, the API returns an identity prediction for 45.2% of the faces in the dataset. Identities in the dataset for videos airing prior to Jan. 1, 2019 result from Celebrity Recognition API queries performed in September 2019. Identities for subsequent videos result from API queries made within a few days of the video's original air date. Amazon does not disclose the full list of individuals that can be recognized by the Amazon Rekognition Celebrity Recognition API.
To increase the percentage of faces with identity tags, we also add identity tags to faces that were not identified by the Celebrity Recognition API, but have close visual similarity to identified faces. This is accomplished with a nearest neighbor classifier on a per video basis; we take the faces that have no identity label provided by the Celebrity Recognition API, and find all of their neighbors in face embedding space within a given L2 distance, and then assign the unlabeled face the majority vote of those labels. In total, 55.5% of the faces in the dataset contain an identity tag. However, the Stanford Cable TV News Analyzer limits use of face screen time filters to individuals that have at least ten hours of estimated screen time as of Aug. 1, 2020.
For each face tagged with an identity, we download all person-tags associated with this identity from DBpedia [5] using its public SPARQL API. A full description of the per-face tags available in the Stanford Cable TV News Analyzer is available on the tag information page. Tags listed with source of "DBpedia" on that page were directly downloaded from DBpedia.
We also augment tags downloaded from DBpedia with additional "derived" tags computed from DBpedia tags. For example, the derived tag "african_american" is formed from the union of DBpedia tags featuring the prefix "african_american": e.g., "african_american_academics", "african_american_mayors".
We detect commercial segments using an algorithm that scans videos for sequences of black frames (which typically indicate the start and end of commercials) and for video segments where caption text that is either missing or lower case. (We observed that these caption features are indicative of commercials in most videos in our dataset.) Source code for this algorithm is available at: https://github.com/scanner-research/esper-tv/blob/master/app/esper/commercial_detection_rekall.py. The algorithm is written using Rekall, an API for complex event detection in video.
From Jan. 2020 to Jul. 2019, there are 70,559 hours of detected commercials in the dataset, leaving 182,896 hours of program content. By default, video segments lying within commercials are excluded from query results in the Stanford Cable TV News Analyzer.
Validation: We performed human annotation of the location of commercial segments in a 20-hour sampling of the dataset. (6.3 hours of this time fell within commercial segments.) On this test set, the commercial detector achieves a precision of 93.0% and a recall of 96.8%. Precision and recall are computed based on per-frame commercial detector results. Specifically, the precision of commercial detection is computed as the faction of frames that the classifier correctly classified as commercials divided by the total number of frames the classifier estimated were part of a commercial segment.
We use the Gentle word aligner to perform sub-second alignment of words in a video's closed-caption caption to the video's audio track. (The source captions are only coarsely aligned to the video.) To perform alignment, we partition the video's audio track into one-second chunks and use Gentle to search for word-level alignment with caption text within +/- 10 seconds of this audio segment.
The Stanford Cable TV News Analyzer tabulates the screen time of caption-text queries using the duration of words determined via caption time alignment. For example, an utterance of the word "politics" that begins at 10:35.10 in a video and ends at 10:35.90 contributes 0.8 seconds to the estimate of the word's total screen time in the video. Since the Stanford Cable TV News Analyzer tabulates the screen time of caption words matching a query, the screen time estimate for a longer word (e.g., "Mississippi") will likely be greater than that of a shorter word (e.g., "Iraq") even if the two words are spoken the same number of times.
[1] MTCNN Source Page (https://kpzhang93.github.io/MTCNN_face_detection_alignment/index.html)
[2] Zhang, Kaipeng, et al. "Joint face detection and alignment using multitask cascaded convolutional networks." IEEE Signal Processing Letters 23.10 (2016): 1499-1503.
[3] FaceNet TensorFlow Implementation (https://github.com/davidsandberg/facenet)
[4] F. Schroff, D. Kalenichenko and J. Philbin, "FaceNet: A unified embedding for face recognition and clustering," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, 2015, pp. 815-823. doi: 10.1109/CVPR.2015.7298682 URL: http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7298682&isnumber=7298593
[5] Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. 2007. DBpedia: a nucleus for a web of open data. In Proceedings of the 6th international The semantic web and 2nd Asian conference on Asian semantic web conference (ISWC'07/ASWC'07).


DWR has a deliberately small Java API. Usually, as a web application developer you will only need to depend on WebContext
/ WebContextFactory
if you are using reverse ajax, you may in addition need ServerContext
/ ServerContextFactory
. These classes are located in the org.directwebremoting
package.
WebContext
and ServerContext
are your connection to other parts of DWR and the general servlet API. WebContext
is used when you are in a DWR thread. It uses a ThreadLocal variable to connect you to the running container. ServerContext
can be used outside of a DWR thread, but as a result you will need to pass it a ServletContext so it knows where DWR is running.
These classes give you to access to the standard HTTP servlet objects:
You use WebContext
like this:
import org.directwebremoting.WebContext; import org.directwebremoting.WebContextFactory; /// WebContext ctx = WebContextFactory.get(); req = ctx.getHttpServletRequest();
It is important that you treat the HTTP request and response as read-only. While HTTP headers might get through OK, there is a good chance that some browsers will ignore them. Any attempt to change the HTTP body WILL cause DWR errors.
See also the JavaDoc for DWR in general, or the specific page for WebContext.
It is possible to get access to the HTTP servlet objects without writing code that depends on DWR - just have the needed parameter (i.e. HttpServletRequest, HttpServletResponse, HttpSession, ServletContext or ServletConfig) declared on your method. DWR will not include it on the generated stub and upon a call of the method it will fill it in automagically.
For example if you have remoted a class like this:
public class Remote { public void method(int param, ServletContext cx, String s) { ... } }
Then you will be able to access it from Javascript just as though the ServletContext
parameter was not there:
Remote.method(42, "test", callback);
DWR will do the work of filling in the parameter for you.
There is one slight caveat with this method. You should ensure you are not using the 'callback function as first parameter' idiom, instead use the 'callback as last parameter' or 'callback in meta-data object' idioms. See the scripting introduction.


public abstract class GlyfDescript extends java.lang.Object implements GlyphDescription
| Modifier and Type | Field and Description |
|---|---|
static byte |
ON_CURVE
if set, the point is on the curve.
|
static byte |
REPEAT
if set, the next byte specifies the number of additional
times this set of flags is to be repeated.
|
static byte |
X_DUAL
This flag as two meanings, depending on how the
x-short vector flags is set.
|
static byte |
X_SHORT_VECTOR
if set, the x-coordinate is 1 byte long.
|
static byte |
Y_DUAL
This flag as two meanings, depending on how the
y-short vector flags is set.
|
static byte |
Y_SHORT_VECTOR
if set, the y-coordinate is 1 byte long.
|
| Modifier | Constructor and Description |
|---|---|
protected |
GlyfDescript(short numberOfContours,
TTFDataStream bais)
Constructor.
|
| Modifier and Type | Method and Description |
|---|---|
int |
getContourCount()
Returns the number of contours.
|
int[] |
getInstructions()
Returns the hinting instructions.
|
protected void |
readInstructions(TTFDataStream bais,
int count)
Read the hinting instructions.
|
void |
resolve()
Resolve all parts of an composite glyph.
|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
getEndPtOfContours, getFlags, getPointCount, getXCoordinate, getYCoordinate, isComposite
public static final byte ON_CURVE
public static final byte X_SHORT_VECTOR
public static final byte Y_SHORT_VECTOR
public static final byte REPEAT
public static final byte X_DUAL
public static final byte Y_DUAL
protected GlyfDescript(short numberOfContours, TTFDataStream bais) throws java.io.IOException
numberOfContours
- the number of contoursbais
- the stream to be readjava.io.IOException
- is thrown if something went wrongpublic void resolve()
resolve
in interface GlyphDescription
public int getContourCount()
getContourCount
in interface GlyphDescription
public int[] getInstructions()
protected void readInstructions(TTFDataStream bais, int count) throws java.io.IOException
bais
- the stream to be readcount
- the number of instructions to be readjava.io.IOException
- is thrown if something went wrong


Gene integrates information from a wide range of species. A record may include nomenclature, Reference Sequences (RefSeqs), maps, pathways, variations, phenotypes, and links to genome-, phenotype-, and locus-specific resources worldwide.
Source: Extract from Gene's home page [2018/10/26]
This plugin was not developed by the NCBI Gene team. NCBI Gene neither supports nor endorses this work. References to NCBI Gene in this plugin are meant only to credit the source of data.
Example queries: 'PTPN2', 'interferon receptor'


{% extends 'base.html' %} {% block head_title %} Add a Server {% endblock %} {% block content %}
Please wait this may take a few minutes...
#}Enter your domain administrator credentials for your Active Directory Server to begin the audit!
{% endblock %}


--- layout: archive title: Zulip Chat Archive permalink: /stream/113489-new-members/topic/Product.20types.html ---
I need to define a type of the form (A x B) where B might differ depending on what term of A I pick. In my case I want to define something where the terms are of the form: ((Y : C), (f : Z ⟶ Y))
Where (Z : C)
where C is a category and the arrow is hom. How would I go about doing this (is this even possible)?
hom is defined as a function:
class has_hom (obj : Type u) : Type (max u (v+1)) := (hom : obj → obj → Type v)
So I thought that I would be OK to define it as a subtype of (C x Type v)
but this wont work as I came to realise my terms will have type Z ⟶ Y
not Type v which is the type of Z ⟶ Y
.
That's a Sigma
type. It could also be done with a structure which would be preferable.
Thank you! I thought Sigma
only could be used when a function had a "product" output where the 2nd type depended on the 1st.
I will try to define a structure then
{% endraw %}Last updated: Apr 24 2020 at 19:02 UTC


No. 18 DePaul women top Miami (Ohio) 98-79 in preseason WNIT
CHICAGO (AP) - Chante Stonewall had 23 points and 12 rebounds, Jolene Daninger scored 14 points and No. 18 DePaul beat Miami (Ohio) 98-79 on Friday night in the preseason WNIT.
Stonewall, the Big East tournament’s most outstanding player last year, scored six points during DePaul’s 16-1 run to start the second quarter for a 44-14 lead. She finished 11 of 14 of the field in just 26 minutes as DePaul shot just 43.6 percent, but outrebounded Miami 53-35.
Sonya Morris added 12 points and Lexi Held and Deja Church, a Michigan transfer, each scored 11 for DePaul. Coach Doug Bruno is five wins away from reaching the 700 plateau, joining Connecticut’s Geno Auriemma and Villanova’s Harry Perretta as the only active coaches to do so.
Savannah Kluesner led Miami with 21 points and Peyton Scott added 15 in the season opener for both teams.
More AP college basketball: https://apnews.com/Collegebasketball and https://twitter.com/AP_Top25
Sign up for Daily Newsletters
Copyright © 2019 The Washington Times, LLC.
Please read our comment policy before commenting.


1
2
Define Catchment Area:
Click on the map defines the vertices of a polygon. Create a closed polygon that represents the hospitals catchment area.
(Note: No edges can cross over)
Click on the map defines the vertices of a polygon. Create a closed polygon that represents the hospitals catchment area.
(Note: No edges can cross over)
3
Maroondah Hospital
Alfred Hospital


Two days ago, I had my first Salvia trip, and I can honestly say that I was completely blown away. I was with 3 relatively close friends and one of them, S, had recently purchased 1 oz of dried Salvia leaves, and 1 gram of the 5 and 10x extract, and we had decided that there was no better opportunity than the present to try something out for the first time.
We started by smoking a bowl of the leaves, sprinkled with a dash of the 5x, and that was interesting, sort of a mushroom-like stone for about 15 minutes, nothing too intense. S and myself were both very educated about the effects, smoking method(s), duration, after-effects, etc, of Salvia and we both wanted to see what this plant could really do. We decided on jumping into the 10x and see what would happen. We each took turns, S went first and myself went second. After I had seen the look on S's face during his trip, I knew that I was in for a ride. I had been looking forward to this experience for quite a while, and the entire day I was in a fantastic mood and was ready for whatever Lady Salvia was ready to throw at me.
I took my first hit and held it in until I counted my way from 1 to 45. I blew out the smoke and my friends each said 'There's no smoke left man' and I replied with a simple 'Sweet'. I took my second and final hit. After I took the hit, I was holding it in and was trying to count up to just 25 this time. But as I was holding it in I was slipping into the grip of this strange plant and my friends had to tell me to exhale.
 I found this to be quite funny for some reason and I started laughing hysterically.  After I don't know how long of watching the strange visuals (it was mostly tracers and odd colors that seemed out of place) and laughing at them, I leaned down on my leg because I felt that some force was making me want to lean down and rest for a moment, before I really took off.
I don't know how long I was laying on my leg for, sitting in the back of a suburban truck, but S leaned me back up because he was getting worried that something was wrong. As he did this I thought that I had fallen down onto the ground and I gasped with shock. I looked between my legs to try and see what the hell was going on and I saw a very angry looking saw coming for me. I blinked my eyes to see if it would go away, because let's face it, watching as a saw is coming closer and closer to my stomach, is a little freaky.
When I opened my eyes, I thought that I was on a subway. When I say that I was on a subway, I don't mean on the inside in a seat, I mean being dragged along for dear life holding onto the side of the door trying to get in. And it actually felt like I was being raced down this dark tunnel actually holding on for dear life. Once again I blinked to make it go away, and opened my eyes to the sight of Coca-Cola bottles surrounded by dancing and twirling candy. This amused me. And as I was trying to enjoy the lovely visuals, I looked down at my feet, because it felt like something was grabbing at them. I ended up seeing this incredibly frightening rip that was going down the middle of the truck. That did it for me, I was terrified. I had to get out of there.
So I abruptly got up and ran outside to get away, because I didn't want to know, at that time, what was on the other side of that tear through the supposed universe. As soon as I got outside, my vision cleared, I could actually tell who my friends were again, I remembered that I had smoked the Salvia and that I was gonna be ok. I then sat back down in the truck and waited for the plant's effects to completely wear off.
 I'm an experienced psychoactive and psychedelic user and this was much more intense than any acid trip I've ever had. On acid, I find that I can ease myself into what is going on. I know that I had taken the acid, I know to enjoy the show, and I can let it work its magic and tickle me and my senses, but with Salvia, it's like being hit in the back of the head with a sledgehammer. I can't remember the last time that I was in awe of a substance.
One of the most interesting things about Salvia to me is that it only lasts for maybe 10 or 15 minutes, how can my mind be totally torn apart and then reassembled in only 10 minutes? I know that I have only scratched the surface with Salvia, and am excited to be able to try it again, only in a much better setting than the back of an SUV in the woods. I think that Salvia has much potential for me and my psychoactive use. I highly recommend Salvia to any psychonaut who is willing to put their mind through a very serious ass-kicking.
Experience Reports are the writings and opinions of the individual authors who submit them.
Some of the activities described are dangerous and/or illegal and none are recommended by Erowid.


Pig Dice
It's
's turn
Click to Roll
Click to Hold
Click to Start the Game
new player name
new player number
enter a player


Gateway
BOT
06-Jun-19 01:15 AM
[dscrd-hush] <dukeleto> @Flash I am not sure exactly what you are doing. You need to move all funds that touched agama to a new address not controlled or touched by agama. The web wallet is the easiest and is a good temp solution
[dscrd-hush] <dukeleto> see pinned posts, too
[dscrd-hush] <dukeleto> I don't lightly recommend web wallets, but the web wallet is considered safer than Agama currently, by a long shot
[dscrd-hush] <Flash> @dukeleto I used the web wallet to create new addresses and just asked what else I should do. Guide said web wallet was not best solution for large holding gs
[dscrd-hush] <dukeleto> true, but Agama is trying to steal your money, so that is worse
[dscrd-hush] <dukeleto> @Flash you need to send funds from Agama to your new web wallet address starting with R...
[dscrd-hush] <dukeleto> the web wallet gives you a bunch of addresses by default, I think 10 or 20 ?
[dscrd-hush] <Flash> My agama is from Feb 2018, I had not logged into it since can't remember. Is it still current
[dscrd-hush] <dukeleto> which version?
[dscrd-hush] <dukeleto> your old agama might not be able to connect to current servers, unfortunately
[dscrd-hush] <Flash> Trying to load up now. Looks like v<IP_ADDRESS>c
[dscrd-hush] <dukeleto> not going to work
[dscrd-hush] <dukeleto> did you use it in lite mode or full mode?
[dscrd-hush] <Flash> So by setting up my new private keys with the new R addresses I am still at risk
[dscrd-hush] <dukeleto> You need to move funds out of agama or you are still at risk
[dscrd-hush] <Flash> I want to say lite mode
[dscrd-hush] <dukeleto> yes, lite mode is most common
[dscrd-hush] <dukeleto> ok, you want to use Verus Agama I believe
[dscrd-hush] <Flash> Okay, I can download and start the move
[dscrd-hush] <dukeleto> @Flash look at our pinned posts, download the newest Verus agama, and send all funds to that new address from Verus agama (there are also 2 other agama versions that would work, but Verus agama seems to be the most used)
[dscrd-hush] <dukeleto> your seed phrase from regular Agama will work in Verus Agama, but the whole point is to send all funds out of that address
[dscrd-hush] <Flash> Most used means better chance at support
[dscrd-hush] <dukeleto> so make a new wallet + seed phrase, write down the address, then log out, use your OLD seed phrase, and then send a transaction with ALL FUNDS to your new address. There is a little "all" button you can click to make it easy
[dscrd-hush] <Flash> Do I need to save the .dat file from old agama wallet?
[dscrd-hush] <dukeleto> and make sure to backup your new seed phrase on paper
[dscrd-hush] <dukeleto> @Flash never delete a wallet.dat, just in case. Back them up, maybe give them more useful names, like old_agama_wallet.dat
[dscrd-hush] <dukeleto> Verify that you have swept all funds from all chains
[dscrd-hush] <Flash> Always have backups. Just need to find the .dat file
[dscrd-hush] <dukeleto> good luck, sorry for the pain. Hackers gonna hack, I guess. We are learning a lot from this experience
[dscrd-hush] <dukeleto> this will make GUI wallets in our ecosystem more secure, this incident. Many things will change because of this
[dscrd-hush] <dukeleto> Various other cryptocoins have contacted me, concerned for their own wallets. And they very much have reason to be concerned
[dscrd-hush] <Flash> So if I used the https://wallet.myhush.org/ and created my Hush3 addresses and they showed a balance then I can guess I have not been hacked yet but should proceed immediately with your instructions
[dscrd-hush] <Flash> Also did not find a wallet.dat file for the old Agama, Looked in AppData and found the Agama folder but no wallet.dat
[dscrd-hush] <dukeleto> lite mode wont' have a wallet.dat, sorry for the confusion
[dscrd-hush] <dukeleto> @Flash did you ever use Agama? it's only if the funds were in agama that there is anything to do
[dscrd-hush] <dukeleto> if you just have funds in our web wallet, and never used Agama, you are safe
[dscrd-hush] <Flash> No worries just glad you had a few minutes to chat. I downloaded Agama last year to check my balances, I had to follow directions to add the Hush coin to the wallet and that was it.
[dscrd-hush] <dukeleto> @everyone DO NOT USE https://www.atomicexplorer.com/wallet/, it's been compromised. All your funds will be drained upon logging in . The Hush web wallet is not affected by this
[dscrd-hush] <dukeleto> gosh what a fun week
[dscrd-hush] <Hammerstein> @dukeleto Is Verus dangerous?
[dscrd-hush] <Flash> I just checked my addresses with the Dexstats link and it shows HUSH3 and BTCH balances that are exactly matching my records.
[dscrd-hush] <yefuchen> thanks for the warning
[dscrd-hush] <dukeleto> @Hammerstein in what way?
[dscrd-hush] <Hammerstein> @dukeleto Unsafe to use and/or leave funds in?
[dscrd-hush] <dukeleto> @Hammerstein I don't trust Agama source code much anymore, regardless of which project has it. Currently Verus Agama is considered safe, but that could change. Full node wallets are safest, and those that don't use nodeJS are much safer
[dscrd-hush] <Flash> @dukeleto is superman on this save your Hush front. So glad to see the engagement and concern
[dscrd-hush] <dukeleto> Just trying to do my best to help people protect their funds
[dscrd-hush] <dukeleto> it's pretty much my job, I think
[dscrd-hush] <dukeleto> The Hush web wallet works differently than the KMD web wallet
[dscrd-hush] <dukeleto> probably a good time for a story around the fire
[dscrd-hush] <dukeleto> KMD web wallet talks to electrum backend servers, but HUSH web wallet talks to our insight explorer API
[dscrd-hush] <dukeleto> they are actually very very different codebases
[dscrd-hush] <dukeleto> So that, at least, makes me sleep better
[dscrd-hush] <dukeleto> the HUSH web wallet does use Javascript tho, it's not immune to this kind of hack
[dscrd-hush] <dukeleto> it changes very very rarely, is one thing it's got going for it
[dscrd-hush] <dukeleto> Hackers go for codebases that constantly change, because that is where you are most likely to get your malware code added
[dscrd-hush] <dukeleto> They show up as a helpful github contributor, with a new package that solves some bug. Or they actually solve a bug and use their own malware to "solve" it. Once a few projects use this malware, the attacker begins to plan the attack. The attack is time sensitive, because it gives up the game and the attacker will probably have to get their code into a new project all over again. Attackers try very hard and use a lot of energy to not get caught, because getting caught is their biggest source of work. It makes them find a whole new project to inject their malware
[dscrd-hush] <dukeleto> @Flash fuck, yeah we should change that
[dscrd-hush] <dukeleto> @gilardh can you please remove the agama section from our website?
[dscrd-hush] <dukeleto> @Flash thanks for the reminder
[dscrd-hush] <Flash> I can create a new wallet on the Webwallet and transfer all my funds from old to new as well right?
[dscrd-hush] <dukeleto> @Flash yes
[dscrd-hush] <Flash> sweet that is going to be by first line of defense until I can get the full node up and runnning
[dscrd-hush] <dukeleto> that sounds great
[dscrd-hush] <dukeleto> we will be increasing our documentation about setting up a full node wallet after all this madness
[dscrd-hush] <dukeleto> I know it's not as easy as it could be. We are striving to make it easier


참고
Click here to download the full example code
Introduction || Tensors || Autograd || Building Models || TensorBoard Support || Training Models || Model Understanding
PyTorch TensorBoard Support¶
Follow along with the video below or on youtube.
Before You Start¶
To run this tutorial, you’ll need to install PyTorch, TorchVision, Matplotlib, and TensorBoard.
With conda
:
conda install pytorch torchvision -c pytorch
conda install matplotlib tensorboard
With pip
:
pip install torch torchvision matplotlib tensorboard
Once the dependencies are installed, restart this notebook in the Python environment where you installed them.
Introduction¶
In this notebook, we’ll be training a variant of LeNet-5 against the Fashion-MNIST dataset. Fashion-MNIST is a set of image tiles depicting various garments, with ten class labels indicating the type of garment depicted.
# PyTorch model and training necessities
import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
# Image datasets and image manipulation
import torchvision
import torchvision.transforms as transforms
# Image display
import matplotlib.pyplot as plt
import numpy as np
# PyTorch TensorBoard support
from torch.utils.tensorboard import SummaryWriter
Showing Images in TensorBoard¶
Let’s start by adding sample images from our dataset to TensorBoard:
# Gather datasets and prepare them for consumption
transform = transforms.Compose(
[transforms.ToTensor(),
transforms.Normalize((0.5,), (0.5,))])
# Store separate training and validations splits in ./data
training_set = torchvision.datasets.FashionMNIST('./data',
download=True,
train=True,
transform=transform)
validation_set = torchvision.datasets.FashionMNIST('./data',
download=True,
train=False,
transform=transform)
training_loader = torch.utils.data.DataLoader(training_set,
batch_size=4,
shuffle=True,
num_workers=2)
validation_loader = torch.utils.data.DataLoader(validation_set,
batch_size=4,
shuffle=False,
num_workers=2)
# Class labels
classes = ('T-shirt/top', 'Trouser', 'Pullover', 'Dress', 'Coat',
'Sandal', 'Shirt', 'Sneaker', 'Bag', 'Ankle Boot')
# Helper function for inline image display
def matplotlib_imshow(img, one_channel=False):
if one_channel:
img = img.mean(dim=0)
img = img / 2 + 0.5 # unnormalize
npimg = img.numpy()
if one_channel:
plt.imshow(npimg, cmap="Greys")
else:
plt.imshow(np.transpose(npimg, (1, 2, 0)))
# Extract a batch of 4 images
dataiter = iter(training_loader)
images, labels = next(dataiter)
# Create a grid from the images and show them
img_grid = torchvision.utils.make_grid(images)
matplotlib_imshow(img_grid, one_channel=True)
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-images-idx3-ubyte.gz to ./data/FashionMNIST/raw/train-images-idx3-ubyte.gz
0%| | 0/26421880 [00:00<?, ?it/s]
0%| | 32768/26421880 [00:00<03:05, 141948.22it/s]
0%| | 65536/26421880 [00:00<03:06, 141505.31it/s]
0%| | 131072/26421880 [00:00<02:13, 197286.03it/s]
1%| | 196608/26421880 [00:00<01:54, 229427.76it/s]
1%| | 262144/26421880 [00:01<01:45, 249030.96it/s]
1%|1 | 327680/26421880 [00:01<01:40, 260298.75it/s]
2%|1 | 425984/26421880 [00:01<01:22, 313295.74it/s]
2%|1 | 524288/26421880 [00:01<01:14, 347433.14it/s]
2%|2 | 589824/26421880 [00:02<01:18, 328106.23it/s]
3%|2 | 688128/26421880 [00:02<01:13, 350898.98it/s]
3%|2 | 786432/26421880 [00:02<01:08, 373070.98it/s]
3%|3 | 917504/26421880 [00:02<00:59, 429647.07it/s]
4%|3 | 1015808/26421880 [00:03<00:59, 428423.86it/s]
4%|4 | 1146880/26421880 [00:03<00:53, 469530.27it/s]
5%|4 | 1245184/26421880 [00:03<00:54, 458069.88it/s]
5%|5 | 1376256/26421880 [00:03<00:51, 489786.94it/s]
6%|5 | 1507328/26421880 [00:03<00:48, 512630.78it/s]
6%|6 | 1671168/26421880 [00:04<00:43, 568790.27it/s]
7%|6 | 1802240/26421880 [00:04<00:43, 569414.60it/s]
7%|7 | 1966080/26421880 [00:04<00:40, 608811.79it/s]
8%|8 | 2129920/26421880 [00:04<00:38, 638033.89it/s]
9%|8 | 2293760/26421880 [00:05<00:36, 660314.58it/s]
9%|9 | 2490368/26421880 [00:05<00:33, 714469.52it/s]
10%|# | 2686976/26421880 [00:05<00:31, 752480.42it/s]
11%|# | 2883584/26421880 [00:05<00:30, 783075.68it/s]
12%|#1 | 3080192/26421880 [00:06<00:29, 802163.23it/s]
13%|#2 | 3309568/26421880 [00:06<00:26, 858322.55it/s]
13%|#3 | 3538944/26421880 [00:06<00:25, 897387.21it/s]
14%|#4 | 3768320/26421880 [00:06<00:24, 927241.21it/s]
15%|#5 | 4030464/26421880 [00:06<00:22, 985476.60it/s]
16%|#6 | 4292608/26421880 [00:07<00:21, 1031415.34it/s]
17%|#7 | 4587520/26421880 [00:07<00:19, 1100615.64it/s]
18%|#8 | 4882432/26421880 [00:07<00:18, 1151906.33it/s]
20%|#9 | 5177344/26421880 [00:07<00:17, 1190757.04it/s]
21%|## | 5505024/26421880 [00:08<00:16, 1254085.31it/s]
22%|##2 | 5832704/26421880 [00:08<00:15, 1305196.12it/s]
23%|##3 | 6193152/26421880 [00:08<00:14, 1379527.56it/s]
25%|##4 | 6586368/26421880 [00:08<00:13, 1470262.48it/s]
26%|##6 | 6979584/26421880 [00:09<00:12, 1541437.74it/s]
28%|##8 | 7405568/26421880 [00:09<00:11, 1630199.48it/s]
30%|##9 | 7864320/26421880 [00:09<00:10, 1729755.89it/s]
32%|###1 | 8323072/26421880 [00:09<00:08, 2041838.98it/s]
32%|###2 | 8552448/26421880 [00:09<00:08, 2078927.35it/s]
33%|###3 | 8847360/26421880 [00:10<00:09, 1823876.66it/s]
36%|###5 | 9404416/26421880 [00:10<00:08, 2024294.76it/s]
38%|###7 | 9994240/26421880 [00:10<00:07, 2195463.28it/s]
40%|#### | 10616832/26421880 [00:10<00:06, 2295802.69it/s]
43%|####2 | 11304960/26421880 [00:10<00:06, 2506465.95it/s]
46%|####5 | 12025856/26421880 [00:11<00:05, 2679662.47it/s]
48%|####8 | 12779520/26421880 [00:11<00:04, 2860088.67it/s]
51%|#####1 | 13565952/26421880 [00:11<00:03, 3306901.17it/s]
55%|#####4 | 14417920/26421880 [00:11<00:03, 3424185.76it/s]
58%|#####8 | 15335424/26421880 [00:12<00:03, 3295589.27it/s]
62%|######1 | 16285696/26421880 [00:12<00:02, 3529336.50it/s]
65%|######5 | 17268736/26421880 [00:12<00:02, 4319348.79it/s]
67%|######7 | 17760256/26421880 [00:12<00:02, 4268530.47it/s]
69%|######9 | 18350080/26421880 [00:12<00:01, 4172008.13it/s]
74%|#######3 | 19464192/26421880 [00:12<00:01, 4701596.48it/s]
76%|#######5 | 19955712/26421880 [00:13<00:01, 4567185.38it/s]
78%|#######8 | 20676608/26421880 [00:13<00:01, 4106764.12it/s]
83%|########2 | 21921792/26421880 [00:13<00:00, 4582263.88it/s]
88%|########7 | 23232512/26421880 [00:13<00:00, 5458738.04it/s]
91%|######### | 24018944/26421880 [00:13<00:00, 5904324.21it/s]
93%|#########3| 24674304/26421880 [00:13<00:00, 4972751.80it/s]
99%|#########8| 26083328/26421880 [00:14<00:00, 6044917.75it/s]
100%|##########| 26421880/26421880 [00:14<00:00, 1857788.89it/s]
Extracting ./data/FashionMNIST/raw/train-images-idx3-ubyte.gz to ./data/FashionMNIST/raw
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/train-labels-idx1-ubyte.gz to ./data/FashionMNIST/raw/train-labels-idx1-ubyte.gz
0%| | 0/29515 [00:00<?, ?it/s]
100%|##########| 29515/29515 [00:00<00:00, 62039.23it/s]
100%|##########| 29515/29515 [00:00<00:00, 61932.34it/s]
Extracting ./data/FashionMNIST/raw/train-labels-idx1-ubyte.gz to ./data/FashionMNIST/raw
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-images-idx3-ubyte.gz to ./data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz
0%| | 0/4422102 [00:00<?, ?it/s]
1%| | 32768/4422102 [00:00<00:33, 129548.05it/s]
1%|1 | 65536/4422102 [00:00<00:33, 129125.93it/s]
3%|2 | 131072/4422102 [00:00<00:22, 187886.77it/s]
5%|5 | 229376/4422102 [00:01<00:15, 262781.63it/s]
10%|# | 458752/4422102 [00:01<00:08, 491889.46it/s]
16%|#6 | 720896/4422102 [00:01<00:05, 674016.09it/s]
26%|##5 | 1146880/4422102 [00:01<00:03, 992995.86it/s]
36%|###5 | 1572864/4422102 [00:02<00:02, 1214920.85it/s]
42%|####2 | 1867776/4422102 [00:02<00:02, 1192401.19it/s]
50%|####9 | 2195456/4422102 [00:02<00:01, 1216527.48it/s]
57%|#####7 | 2523136/4422102 [00:02<00:01, 1238829.57it/s]
64%|######4 | 2850816/4422102 [00:03<00:01, 1260372.58it/s]
72%|#######1 | 3178496/4422102 [00:03<00:00, 1278639.33it/s]
80%|######## | 3538944/4422102 [00:03<00:00, 1314631.10it/s]
88%|########8 | 3899392/4422102 [00:03<00:00, 1346390.38it/s]
96%|#########6| 4259840/4422102 [00:04<00:00, 1368153.66it/s]
100%|##########| 4422102/4422102 [00:04<00:00, 1076213.95it/s]
Extracting ./data/FashionMNIST/raw/t10k-images-idx3-ubyte.gz to ./data/FashionMNIST/raw
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz
Downloading http://fashion-mnist.s3-website.eu-central-1.amazonaws.com/t10k-labels-idx1-ubyte.gz to ./data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz
0%| | 0/5148 [00:00<?, ?it/s]
100%|##########| 5148/5148 [00:00<00:00, 22032935.71it/s]
Extracting ./data/FashionMNIST/raw/t10k-labels-idx1-ubyte.gz to ./data/FashionMNIST/raw
Above, we used TorchVision and Matplotlib to create a visual grid of a
minibatch of our input data. Below, we use the add_image()
call on
SummaryWriter
to log the image for consumption by TensorBoard, and
we also call flush()
to make sure it’s written to disk right away.
# Default log_dir argument is "runs" - but it's good to be specific
# torch.utils.tensorboard.SummaryWriter is imported above
writer = SummaryWriter('runs/fashion_mnist_experiment_1')
# Write image data to TensorBoard log dir
writer.add_image('Four Fashion-MNIST Images', img_grid)
writer.flush()
# To view, start TensorBoard on the command line with:
# tensorboard --logdir=runs
# ...and open a browser tab to http://localhost:6006/
If you start TensorBoard at the command line and open it in a new browser tab (usually at localhost:6006), you should see the image grid under the IMAGES tab.
Graphing Scalars to Visualize Training¶
TensorBoard is useful for tracking the progress and efficacy of your training. Below, we’ll run a training loop, track some metrics, and save the data for TensorBoard’s consumption.
Let’s define a model to categorize our image tiles, and an optimizer and loss function for training:
class Net(nn.Module):
def __init__(self):
super(Net, self).__init__()
self.conv1 = nn.Conv2d(1, 6, 5)
self.pool = nn.MaxPool2d(2, 2)
self.conv2 = nn.Conv2d(6, 16, 5)
self.fc1 = nn.Linear(16 * 4 * 4, 120)
self.fc2 = nn.Linear(120, 84)
self.fc3 = nn.Linear(84, 10)
def forward(self, x):
x = self.pool(F.relu(self.conv1(x)))
x = self.pool(F.relu(self.conv2(x)))
x = x.view(-1, 16 * 4 * 4)
x = F.relu(self.fc1(x))
x = F.relu(self.fc2(x))
x = self.fc3(x)
return x
net = Net()
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(net.parameters(), lr=0.001, momentum=0.9)
Now let’s train a single epoch, and evaluate the training vs. validation set losses every 1000 batches:
print(len(validation_loader))
for epoch in range(1): # loop over the dataset multiple times
running_loss = 0.0
for i, data in enumerate(training_loader, 0):
# basic training loop
inputs, labels = data
optimizer.zero_grad()
outputs = net(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
running_loss += loss.item()
if i % 1000 == 999: # Every 1000 mini-batches...
print('Batch {}'.format(i + 1))
# Check against the validation set
running_vloss = 0.0
net.train(False) # Don't need to track gradents for validation
for j, vdata in enumerate(validation_loader, 0):
vinputs, vlabels = vdata
voutputs = net(vinputs)
vloss = criterion(voutputs, vlabels)
running_vloss += vloss.item()
net.train(True) # Turn gradients back on for training
avg_loss = running_loss / 1000
avg_vloss = running_vloss / len(validation_loader)
# Log the running loss averaged per batch
writer.add_scalars('Training vs. Validation Loss',
{ 'Training' : avg_loss, 'Validation' : avg_vloss },
epoch * len(training_loader) + i)
running_loss = 0.0
print('Finished Training')
writer.flush()
2500
Batch 1000
Batch 2000
Batch 3000
Batch 4000
Batch 5000
Batch 6000
Batch 7000
Batch 8000
Batch 9000
Batch 10000
Batch 11000
Batch 12000
Batch 13000
Batch 14000
Batch 15000
Finished Training
Switch to your open TensorBoard and have a look at the SCALARS tab.
Visualizing Your Model¶
TensorBoard can also be used to examine the data flow within your model.
To do this, call the add_graph()
method with a model and sample
input. When you open
# Again, grab a single mini-batch of images
dataiter = iter(training_loader)
images, labels = next(dataiter)
# add_graph() will trace the sample input through your model,
# and render it as a graph.
writer.add_graph(net, images)
writer.flush()
When you switch over to TensorBoard, you should see a GRAPHS tab. Double-click the “NET” node to see the layers and data flow within your model.
Visualizing Your Dataset with Embeddings¶
The 28-by-28 image tiles we’re using can be modeled as 784-dimensional
vectors (28 * 28 = 784). It can be instructive to project this to a
lower-dimensional representation. The add_embedding()
method will
project a set of data onto the three dimensions with highest variance,
and display them as an interactive 3D chart. The add_embedding()
method does this automatically by projecting to the three dimensions
with highest variance.
Below, we’ll take a sample of our data, and generate such an embedding:
# Select a random subset of data and corresponding labels
def select_n_random(data, labels, n=100):
assert len(data) == len(labels)
perm = torch.randperm(len(data))
return data[perm][:n], labels[perm][:n]
# Extract a random subset of data
images, labels = select_n_random(training_set.data, training_set.targets)
# get the class labels for each image
class_labels = [classes[label] for label in labels]
# log embeddings
features = images.view(-1, 28 * 28)
writer.add_embedding(features,
metadata=class_labels,
label_img=images.unsqueeze(1))
writer.flush()
writer.close()
Now if you switch to TensorBoard and select the PROJECTOR tab, you should see a 3D representation of the projection. You can rotate and zoom the model. Examine it at large and small scales, and see whether you can spot patterns in the projected data and the clustering of labels.
For better visibility, it’s recommended to:
Select “label” from the “Color by” drop-down on the left.
Toggle the Night Mode icon along the top to place the light-colored images on a dark background.
Other Resources¶
For more information, have a look at:
PyTorch documentation on torch.utils.tensorboard.SummaryWriter
Tensorboard tutorial content in the PyTorch.org Tutorials
For more information about TensorBoard, see the TensorBoard documentation
Total running time of the script: ( 2 minutes 38.530 seconds)


A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle. This creates an internal state of the network which allows it to exhibit dynamic temporal behavior.
keras.layers.recurrent.SimpleRNN(units, activation='tanh', use_bias=True,
kernel_initializer='glorot_uniform',
recurrent_initializer='orthogonal',
bias_initializer='zeros',
kernel_regularizer=None,
recurrent_regularizer=None,
bias_regularizer=None,
activity_regularizer=None,
kernel_constraint=None, recurrent_constraint=None,
bias_constraint=None, dropout=0.0, recurrent_dropout=0.0)
a(x) = x
).kernel
weights matrix,
used for the linear transformation of the inputs.
(see initializers).recurrent_kernel
weights matrix,
used for the linear transformation of the recurrent state.
(see initializers).kernel
weights matrix
(see regularizer).recurrent_kernel
weights matrix
(see regularizer).kernel
weights matrix
(see constraints).recurrent_kernel
weights matrix
(see constraints).Contrary to feed-forward neural networks, the RNN is characterized by the ability of encoding longer past information, thus very suitable for sequential models. The BPTT extends the ordinary BP algorithm to suit the recurrent neural architecture.
Reference: Backpropagation through Time
%matplotlib inline
import numpy as np
import pandas as pd
import theano
import theano.tensor as T
import keras
import numpy as np
import matplotlib.pyplot as plt
from sklearn.preprocessing import LabelEncoder
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
# -- Keras Import
from keras.models import Sequential
from keras.layers import Dense, Activation
from keras.preprocessing import image
from keras.datasets import imdb
from keras.datasets import mnist
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten
from keras.layers import Conv2D, MaxPooling2D
from keras.utils import np_utils
from keras.preprocessing import sequence
from keras.layers.embeddings import Embedding
from keras.layers.recurrent import LSTM, GRU, SimpleRNN
from keras.layers import Activation, TimeDistributed, RepeatVector
from keras.callbacks import EarlyStopping, ModelCheckpoint
This is a dataset for binary sentiment classification containing substantially more data than previous benchmark datasets.
IMDB provided a set of 25,000 highly polar movie reviews for training, and 25,000 for testing.
There is additional unlabeled data for use as well. Raw text and already processed bag of words formats are provided.
max_features = 20000
maxlen = 100 # cut texts after this number of words (among top max_features most common words)
batch_size = 32
print("Loading data...")
(X_train, y_train), (X_test, y_test) = imdb.load_data(num_words=max_features)
print(len(X_train), 'train sequences')
print(len(X_test), 'test sequences')
print('Example:')
print(X_train[:1])
print("Pad sequences (samples x time)")
X_train = sequence.pad_sequences(X_train, maxlen=maxlen)
X_test = sequence.pad_sequences(X_test, maxlen=maxlen)
print('X_train shape:', X_train.shape)
print('X_test shape:', X_test.shape)
print('Build model...')
model = Sequential()
model.add(Embedding(max_features, 128, input_length=maxlen))
model.add(SimpleRNN(128))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
# try using different optimizers and different optimizer configs
model.compile(loss='binary_crossentropy', optimizer='adam')
print("Train...")
model.fit(X_train, y_train, batch_size=batch_size, epochs=1,
validation_data=(X_test, y_test))
A LSTM network is an artificial neural network that contains LSTM blocks instead of, or in addition to, regular network units. A LSTM block may be described as a "smart" network unit that can remember a value for an arbitrary length of time.
Unlike traditional RNNs, an Long short-term memory network is well-suited to learn from experience to classify, process and predict time series when there are very long time lags of unknown size between important events.
keras.layers.recurrent.LSTM(units, activation='tanh', recurrent_activation='hard_sigmoid', use_bias=True,
kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal',
bias_initializer='zeros', unit_forget_bias=True, kernel_regularizer=None,
recurrent_regularizer=None, bias_regularizer=None, activity_regularizer=None,
kernel_constraint=None, recurrent_constraint=None, bias_constraint=None,
dropout=0.0, recurrent_dropout=0.0)
a(x) = x
).kernel
weights matrix,
used for the linear transformation of the inputs.recurrent_kernel
weights matrix,
used for the linear transformation of the recurrent state.bias_initializer="zeros"
.
This is recommended in Jozefowicz et al.kernel
weights matrix.recurrent_kernel
weights matrix.kernel
weights matrix.recurrent_kernel
weights matrix.Gated recurrent units are a gating mechanism in recurrent neural networks.
Much similar to the LSTMs, they have fewer parameters than LSTM, as they lack an output gate.
keras.layers.recurrent.GRU(units, activation='tanh', recurrent_activation='hard_sigmoid', use_bias=True,
kernel_initializer='glorot_uniform', recurrent_initializer='orthogonal',
bias_initializer='zeros', kernel_regularizer=None, recurrent_regularizer=None,
bias_regularizer=None, activity_regularizer=None, kernel_constraint=None,
recurrent_constraint=None, bias_constraint=None,
dropout=0.0, recurrent_dropout=0.0)
print('Build model...')
model = Sequential()
model.add(Embedding(max_features, 128, input_length=maxlen))
# !!! Play with those! try and get better results!
#model.add(SimpleRNN(128))
#model.add(GRU(128))
#model.add(LSTM(128))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Activation('sigmoid'))
# try using different optimizers and different optimizer configs
model.compile(loss='binary_crossentropy', optimizer='adam')
print("Train...")
model.fit(X_train, y_train, batch_size=batch_size,
epochs=4, validation_data=(X_test, y_test))
score, acc = model.evaluate(X_test, y_test, batch_size=batch_size)
print('Test score:', score)
print('Test accuracy:', acc)
This section demonstrates the use of a Convolutional LSTM network.
This network is used to predict the next frame of an artificially generated movie which contains moving squares.
Generate movies with 3
to 7
moving squares inside.
The squares are of shape $1 \times 1$ or $2 \times 2$ pixels, which move linearly over time.
For convenience we first create movies with bigger width and height (80x80
)
and at the end we select a $40 \times 40$ window.
# Artificial Data Generation
def generate_movies(n_samples=1200, n_frames=15):
row = 80
col = 80
noisy_movies = np.zeros((n_samples, n_frames, row, col, 1), dtype=np.float)
shifted_movies = np.zeros((n_samples, n_frames, row, col, 1),
dtype=np.float)
for i in range(n_samples):
# Add 3 to 7 moving squares
n = np.random.randint(3, 8)
for j in range(n):
# Initial position
xstart = np.random.randint(20, 60)
ystart = np.random.randint(20, 60)
# Direction of motion
directionx = np.random.randint(0, 3) - 1
directiony = np.random.randint(0, 3) - 1
# Size of the square
w = np.random.randint(2, 4)
for t in range(n_frames):
x_shift = xstart + directionx * t
y_shift = ystart + directiony * t
noisy_movies[i, t, x_shift - w: x_shift + w,
y_shift - w: y_shift + w, 0] += 1
# Make it more robust by adding noise.
# The idea is that if during inference,
# the value of the pixel is not exactly one,
# we need to train the network to be robust and still
# consider it as a pixel belonging to a square.
if np.random.randint(0, 2):
noise_f = (-1)**np.random.randint(0, 2)
noisy_movies[i, t,
x_shift - w - 1: x_shift + w + 1,
y_shift - w - 1: y_shift + w + 1,
0] += noise_f * 0.1
# Shift the ground truth by 1
x_shift = xstart + directionx * (t + 1)
y_shift = ystart + directiony * (t + 1)
shifted_movies[i, t, x_shift - w: x_shift + w,
y_shift - w: y_shift + w, 0] += 1
# Cut to a 40x40 window
noisy_movies = noisy_movies[::, ::, 20:60, 20:60, ::]
shifted_movies = shifted_movies[::, ::, 20:60, 20:60, ::]
noisy_movies[noisy_movies >= 1] = 1
shifted_movies[shifted_movies >= 1] = 1
return noisy_movies, shifted_movies
from keras.models import Sequential
from keras.layers.convolutional import Conv3D
from keras.layers.convolutional_recurrent import ConvLSTM2D
from keras.layers.normalization import BatchNormalization
import numpy as np
from matplotlib import pyplot as plt
%matplotlib inline
We create a layer which take as input movies of shape (n_frames, width, height, channels)
and returns a movie
of identical shape.
seq = Sequential()
seq.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
input_shape=(None, 40, 40, 1),
padding='same', return_sequences=True))
seq.add(BatchNormalization())
seq.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
padding='same', return_sequences=True))
seq.add(BatchNormalization())
seq.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
padding='same', return_sequences=True))
seq.add(BatchNormalization())
seq.add(ConvLSTM2D(filters=40, kernel_size=(3, 3),
padding='same', return_sequences=True))
seq.add(BatchNormalization())
seq.add(Conv3D(filters=1, kernel_size=(3, 3, 3),
activation='sigmoid',
padding='same', data_format='channels_last'))
seq.compile(loss='binary_crossentropy', optimizer='adadelta')
# Train the network
noisy_movies, shifted_movies = generate_movies(n_samples=1200)
seq.fit(noisy_movies[:1000], shifted_movies[:1000], batch_size=10,
epochs=20, validation_split=0.05)
# Testing the network on one movie
# feed it with the first 7 positions and then
# predict the new positions
which = 1004
track = noisy_movies[which][:7, ::, ::, ::]
for j in range(16):
new_pos = seq.predict(track[np.newaxis, ::, ::, ::, ::])
new = new_pos[::, -1, ::, ::, ::]
track = np.concatenate((track, new), axis=0)
# And then compare the predictions
# to the ground truth
track2 = noisy_movies[which][::, ::, ::, ::]
for i in range(15):
fig = plt.figure(figsize=(10, 5))
ax = fig.add_subplot(121)
if i >= 7:
ax.text(1, 3, 'Predictions !', fontsize=20, color='w')
else:
ax.text(1, 3, 'Inital trajectory', fontsize=20)
toplot = track[i, ::, ::, 0]
plt.imshow(toplot)
ax = fig.add_subplot(122)
plt.text(1, 3, 'Ground truth', fontsize=20)
toplot = track2[i, ::, ::, 0]
if i >= 2:
toplot = shifted_movies[which][i - 1, ::, ::, 0]
plt.imshow(toplot)
plt.savefig('imgs/convlstm/%i_animate.png' % (i + 1))