454 Bioinformatics Pipeline
From BayPaulWiki
Contents |
Before you begin!
It is standard procedure to include a run_* script for all of the scripts discussed below. This allows staff to review the files and script parameters that were used. By reviewing previous pipeline directories (such as sff_* and VERSION3 in the g454 project directory) you can view and copy/edit existing run_* scripts. New users require a ".dbconf" and a ".my.cnf" in order to process runs through the pipeline.
Move the 454 data files to shire and backup disk
Bring the sff.txt files into the database where they can be processed and analyzed.
New on the FLX: 1. Backup the Run There is a text file on the Desktop called FLX backup instructions: ssh into FLX <IP_ADDRESS> to get name of dir to be backed up
ssh<EMAIL_ADDRESS>log in and cd to data, find dir name, and logout
move into the "Data to Archive" dir
cd /Volumes/Data\To\Archive/
secure copy the remote files
scp -r adminrig@<IP_ADDRESS>:/data/rundir .
2.Copy the files from "Data to Archive" to the run dir in g454/downloads
Create a dir named the rundate (YYYYMMDD) in g454/downloads cd to the run dir in "Data to Archive" the D_dir scp *.* cholmes@arthur.mbl.edu:~/g454/downloads/run dir cd to the sff dir scp *.sff cholmes@arthur.mbl.edu:~/g454/downloads/run dir
Import Info and Sequences into env454
1. Log on to one of the bioinformatics machines (arthur).
2. Review the Sequencing Request form. This file always contains "SeqReq" in the file name with a .csv extension. It can be viewed with the simple "cat" or "less" command. If the researcher entered 'none' for primer, remove this and leave the cells blank.
3. Check the permissions on the .csv file. If necessary change the group and permissions. To view the permissions on all files in your directory use the command "ls -l"
If the permissions look like this: -rw-rw-r-- 1 cholmes cholmes 969 Oct 19 13:36 GSFLXSeqReq_20071018_DMW.csv they need to be changed to look like this: -rw-rw-r-- 1 cholmes g454 969 Oct 19 13:36 GSFLXSeqReq_20071018_DMW.csv Change them with the following commands: a. chgrp g454 GSFLXSeqReq_20071018_DMW.csv b. chmod 664 GSFLXSeqReq_20071018_DMW.csv
Convert the file to UNIX with mac2unix or dos2unix commands. mac2unix GSFLXSeqReq_20071018_DMW.csv dos2unix GSFLXSeqReq_20071018_DMW.csv
NOTE: If you cannot distinguish between dos and unix formatted files, use the dos2unix command anyway to convert the files. If you import the file into BBedit or Text Wrangler you can see how the file is formatted. Alternatively, you can use the unix command cat -v to tell if a file is MAC versus DOS or UNIX formatted.
4. Check the csv files for having a project description, proper dataset names, or any columns missing data. "less GSFLXSeqReq_20071018_DMW.csv". Also check for duplicate run keys. If there are duplicates, they should be in different regions with different primers. The csv file is ordered alphabetically by run key (Processing starts here). Hilary has written a script that will check for duplicated keys called "dupkeys". It should be run from your run directory.
5. Check the primers. They should be in the following list (not case sensitive):
Bacterial V6-V4 Suite Bacterial V3-V5 Suite Archaeal V6-V4 Suite Eukaryal V9 Suite Archaeal V6 Suite Bacterial V6 Suite bacv6 arcv6 eukv9 topo Relman eukv9_1389 eukv9_1380 v3 v6_dutch
5b. Check the sources. They should be in the following list of current, commonly used Ti amplicon sources:
v6v4 (bacterial) v3v5 (bacterial) v6v4a (archaeal) v9 (eukaryal) v6 (bacterial) v6a (archaeal)
6. Import the project, researcher, run keys, and dataset information with import_flxruninfo. If samples are external, double-check the run date to see if it already exists in run_keys. This will write into tables: projects, run_keys, run_primers
import_flxruninfo -r rundate -f SeqReq_MBJ_GOS_Bv3v5.csv
7. Import the sff files into the database with import_flxrun, import once for each region. This will write into rawseq, rawqual, rawflow, rawflowindex. The ampersand "&" allows the command to run in the background.
import_flxrun -r rundate -i EFWL25X01.sff & import_flxrun -r rundate -i EFWL25X02.sff &
8. Check that the sequences imported fully. Often the database connection is dropped and the import fails before all sequences are imported.
a. determine the number of reads you should have: head sff/*.sff.txt | grep "# of Reads:" b. determine the number of reads that made it into the database: mysql -h jbpcdb.mbl.edu env454 -e "SELECT region, count(*) as seqcnt FROM rawseq WHERE run=20081125 group by region order by region"
*** If the counts for each region do not match the number of reads in each sff file, the data will need to be cleared out and rerun where rundate is the actual date of the run, and # is 1 or 2 (which region to clear out) clear454run -raw -reg # -run rundate and, import that region again (import_flxrun...)
c. If the counts are the same, then run the statistics on the reads (in the downloads/rundate directory) report_trimmingstats -raw rundate
Review the results in trimstats_rundate.txt. Be sure that counts are correct and that all datasets have reasonable numbers of reads (at least several thousand) and that runkeys were not found in the wrong region in numbers > 100.
The report above uses the following queries: If the sameregion = 0, and expectedregion is not 0 and there numerous reads (>40) something may be wrong If the value of the sameregion field is 1, and there are few reads, that is also suspect. select r.region as foundregion, count(r.read_id) as seqcnt, k.run_key as run_key, k.region as expectedregion, dataset, (r.region = k.region) as sameregion from (select substr(sequence, 1, 5) as run_key, region, read_id from rawseq where run='20070802') as r left join run_keys as k using (run_key) where k.run='20070802' group by r.run_key, r.region order by seqcnt desc Same query as above ordered by region and dataset: select k.run_key as run_key,dataset, k.region as expectedregion, count(r.read_id) as seqcnt, r.region as foundregion, (r.region = k.region) as same_region from (select substr(sequence, 1, 5) as run_key, region, read_id from rawseq where run='20081030') as r left join run_keys as k using (run_key) where k.run='20081030' group by r.run_key, r.region order by expectedregion, foundregion, dataset If the value of seqcnt is low for any dataset, it is suspect For Run Keys in only one region (region != 0) select k.run_key, k.region, dataset, count(r.read_id) as seqcnt from run_keys as k left join (select read_id, region, substr(sequence,1,5) as run_key from rawseq where run='20070802') as r using (run_key, region) where k.run='20070802' and k.region !=0 group by k.run_key, k.region ALTERNATIVE: If the Run Key is in both regions (region=0) use: select k.run_key, r.region, dataset, count(r.read_id) as seqcnt from run_keys as k left join (select read_id, region, substr(sequence,1,5) as run_key from rawseq where run='20070823') as r using (run_key) where k.run='20070823' and k.region=0 group by k.run_key, r.region
Trim the sequences
Removes low quality reads and the primers from the high quality reads, and assigns dataset and project names to each read
1. Log on to the cluster (grendel). From the run directory submit clustertrim to the cluster and run: noclustertrim -r rundate You can check for when your processes have completed on the cluster: qstat or : qstat | grep your_username
2. When the trimming is complete, finishing running the statistics (still in the downloads/run directory) report_trimmingstats -trim rundate This will run the following queries: count the total reads SELECT count(*) from trimseq where run=20081120 count reads for each dataset SELECT project, dataset, deleted, count(*) FROM trimseq where run=20081120 and dataset != and project != GROUP BY dataset, deleted ORDER BY dataset, deleted count the reads that were deleted and why SELECT delete_reason, COUNT(*) FROM trimseq WHERE run='20081120' GROUP by delete_reason delete_reason=0 (not deleted) should be much greater than all other reasons. If any delete_reason is greater than 1% (except proximal), the trimming should be investigated before proceeding. If the total is greater than 15 % look at the biggest reason. If biggest delete_reason is 'key' then the regions may be confused try: SELECT substring(s.sequence, 1,5) as firstfive, region, count(t.read_id) as cnt, COUNT(CASE WHEN deleted=0 then t.read_id ELSE NULL END) as cntkept, COUNT(CASE WHEN deleted=1 then t.read_id ELSE NULL END) as cntdeleted FROM `trimseq` as t join rawseq as s using (read_id) WHERE t.run='20070514' and s.run='20070514' GROUP BY region, firstfive ORDER BY cnt desc This query shows the results for all keys. Useful for finding lost keys, i.e. if no data was recovered for a particular key: SELECT substr(sequence,1,5) as rkey, r.region, dataset, count(*) as cnt FROM rawseq as r left join run_keys as k on (r.run = k.run and r.region=k.region and substr(sequence,1,5)=k.run_key) WHERE r.run='20080507' GROUP by rkey, r.region ORDER by cnt desc
GAST - distance to nearest RefHVR references
GAST (global alignment search tool) uses USearch to find a set of top hits for each sequence (unique read) in a reference database of sequences with accepted taxonomy (our RefHVR database), then does a global alignment and distance calculation to find the reference sequence(s) most similar to each 454 sequence. GAST needs to be run separately for each hypervariable target region in the run. For example, if you have both v6 and v9 data, run once for v6, the repeat it for v9.
1. Log onto the cluster.
2. From the gast directory (cd /xraid2-2/g454/gast), run clustergast for each hypervariable region sequenced, i.e.:
clustergast -r 20100929 -s v6v4 clustergast -r 20100929 -s v3v5 clustergast -r 20100929 -s v6v4a
- This script creates a directory (e.g. 20100929_v6v4) with a fasta file of all reads that were not deleted for quality and a second fasta file containing only the unique sequences represented by the reads, with a .names file that links each sequence to the reads it represents. It then splits up the file of sequences and runs the GAST algorithm in parallel, generating many .txt files as output.
3. When the cluster processes have completed, log on to a bioware machine and confirm that all the sequences were upload by comparing the gast output in the *txt files with the contents of the new table in env454:
- Using mysql check the number of reads that were loaded:
mysql -h jbpcdb.mbl.edu env454 -e "SELECT COUNT(distinct read_id) FROM gast_20100929_v6v4"
- Compare the result with the number of gast results in the g454/gast directory for each source you have used:
cat /xraid2-2/g454/gast/20100929_v6v4/gast_20100929_v6v4.*.txt | cut -f1 | sort | uniq | wc -l
- If this does not equal the number of reads in the gast_run table you will need to manually load the remaining data:
cd $g454/gast/20100929_v6v4 for i in gast_20100929_v6v4.*.txt; do mysqlimport -i -C -v -L -h jbpcdb env454 $i; done; Check the count in the gast_20100929_v6v4 table again. It should now match the results in the directory.
4. Check to see how many sequences were assigned a taxonomy by GAST. The number of records in the gast table should be nearly equal to the number of sequences in the fasta file of unique sequences in the gast working directory.
Example: facount /xraid2-2/g454/gast/20100929_v6v4/20100929_v6v4.unique.fa = 481,034 mysql -h jbpcdb.mbl.edu env454 -e "SELECT COUNT(distinct read_id) FROM gast_20100929_v6v4" = 471,803 481,034 / 471,803 ~= 1.02, or 2% larger.
- The difference is the number of unique sequences >=70% different from any sequence in the reference database.
- For bacterial samples, you may drop more or fewer sequences depending on the degree of eukaryotic DNA in your sample.
- You will likely drop more reads for eukaryotic and archaeal projects than for bacterial ones.
- A very large difference indicates something went wrong in assigning taxonomy and you should confirm that your primers are correct and that you really amplified the same region that you told the pipeline you did.
- Note that the gast count will change after doing gast_cleanup below, so this count is only valid at this point in the process.
- If the counts are off, first check that the initial set of sequences is correct
grep -c ">" 20100929_v6v4/*.fa mysql -h jbpcdb.mbl.edu env454 -e "SELECT COUNT(DISTINCT sequence) FROM trimseq WHERE deleted=0 AND run=20100929 AND source = 'v6v4'"
- If anything goes wrong, the process can be restarted, see clustergast usage by typing 'clustergast' on the commandline.
- (Note: you can clear out the table first by: delete from gast_20100929_v6v4; [HGM])
Example of a restart: clustergast -r 20100929 -s v6v4 -start bestalign.
5. Now move the data into the central gast tables:
cd $g454/gast/20100929_v6v4 gast_cleanup -r 20100929 -reg v6v4 &
- This also expands the table by adding back all the reads with the same sequence, and adds all the reads that were deleted for quality (thus the reads are never really deleted, they are just flagged. There is a field in the table for whether the read was deleted: deleted=1 for deleted, deleted=0 for not deleted)
6. Check the results in the gast and gast_concat tables:
gast_check 20100929 v6v4
- which performs the following series of database queries:
- select count(distinct read_id) from gast_20100929_v6v4 where refhvr_id=0 and distance = 1
- which is the total number of reads that were not assigned a taxonomy by gast. This should be a slightly larger number that the number of sequences that were not assigned a taxonomy from (4) above.
- The next four numbers should all be the same. They are the total number of reads in the gast table from (4) above and the number in the main trimseq, gast, and gast_concat tables, found with the following queries:
- select count(read_id) from trimseq where run=20071018 AND deleted=0 and source = 'v6v4';
- select count(distinct read_id) from gast_20109029_v6v4 ;
- select count(distinct g.read_id) from gast as g join trimseq as t using(read_id) where run='20100929' and deleted=0 and source = 'v6v4';
- select count(g.read_id) from gast_concat as g join trimseq as t using(read_id) where run='20100929' and deleted=0 and source = 'v6v4';
- which performs the following series of database queries:
- The gast table has a separate entry for each best match to the reference database for each read, thus the "distinct".
- The gast_concat table has a single entry for each read.
7. Log back onto the cluster and cd to the gast directory corresponding to the run. Expand the taxonomic assignment from the sequences to each read with gast2tax, which stores data the tagtax table.
- Use defaults for the output table, boot scores and majority.
- The reference table should be refhvr_* where * is the vregion from 5' to 3' -- in other words, for v6v4 specify refhvr_v4v6.
clusterize gast2tax -g gast_20100929_v6v4 -reg v6v4 -ref refhvr_v4v6
- To verify, count the reads in the tagtax table from your run. The count should match the sum of the counts from step (6) above.
mysql -h jbpcdb.mbl.edu env454 -e "select count(*) from tagtax as x join trimseq as t using(read_id) where run='20100929' and deleted=0"
- You can get some summary information about the taxonomic diversity of your run with this query:
mysql -h jbpcdb.mbl.edu env454 -e "select taxonomy, count(*) from tagtax as x join trimseq as t using(read_id) where run=20100929 group by taxonomy"
8. When the data are processed, please send an email notification to<EMAIL_ADDRESS>with the subject line
- New VAMPS data ready
- The body of the email should contain:
- 1) the date of the run,
- 2) projects included in the run (from the SeqReq forms)
- 3) and any users requiring access to each project.
- Remember, BPC users automatically have access to all projects, other users are provided access to non-public data on a project by project basis.
Uploading sequences to VAMPS
After the sequences have been trimmed, and the taxonomy run, the data are ready to be uploaded to the VAMPS server. You must have permissions on the VAMPS database to run this.
1. cd to the $g454/vamps/tovamps directory.
2. run "./run_vamps_upload"
3. Much of the script will run in the background, and generally takes 2 or 3 hours to run. When it is complete, the upload needs to be confirmed prior to replacing the existing VAMPS data. Doing a simple row count is generally sufficient -- checking that the new tables have more records than the old. Check tables: vamps_data_cube, vamps_export, vamps_junk_data_cube, vamps_projects_datasets, vamps_sequences and vamps_taxonomy, by getting a record count for each, and comparing this to the table with the same name, although having a "_transfer" suffix.
4. If all tables have the been properly loaded, they can be swapped in, and the vamps_dev website uploaded as well. a. run "./run_vamps_upload_transfer" b. run "./run_vamps_upload_dev"
Clustering Sequences
After the sequences have been trimmed, they need to be clustered for estimating OTUs and calculating diversity. Please do not try this on v6v4 data: check with Sue or David first
1. go to $g454/dotur/vamps
2. Use vamps_dotur_clusters for all new projects: vamps_dotur_clusters - runs DOTUR on all v6 datasets and packages the the data by project and file type for copying to vamps Clusters v3, v6, and v9 projects only Usage: vamps_dotur_clusters [update | all | project] ex: vamps_dotur_clusters update ex: vamps_dotur_clusters project=ICM_DSS_Bv6 Options: update updates only new (not yet included) projects (does not have a directory in $g454/dotur/data/tovamps) all reruns all projects in the database (v3, v6, v9) Run: vamps_dotur_clusters update
3: This creates a series of run_* scripts that should be run in order. a. From the cluster run: run_mkdir_1 run_rarify_2 run_tax_3 b. When the clusters are complete, log onto a bioinformatics machine run_pack_clusters_4 run_scp2vamps_5 run_chmod_6 c. Log onto the vamps server(s) and double-check the permissions in /usr/local/www/vamps/docs/data_downloads/dotur/* The permissions -rw-rw---- should be set by the script, but in case they are not, do: chmod g+w *
Adding new users
For adding new user accounts, [| See the VAMPS wiki. ]
Adding a new project to vamps through the vamps_users table. Do this on vampsdev and vamps
insert into vamps_users (user, project) values ('username' ,'project')
Taxonomy Spreadsheets
Once the taxonomy has been completed in the database, Excel spreadsheets are used to communicate the information to the researchers. This step only needed for v9 data.
1. Copy the template.xls spreadsheet to a new Excel spreadsheet, named for the project not the run, as researchers often do not know the run dates.
2. Go to the SQL tab, and begin editing. The first thing to find is the relative abundance of tags for each dataset. Rerun this SQL and paste the results back into the SQL tab.
3. Edit the 6 taxonomy SQL statements: Update the datasets in the COUNT(CASE statements, Update the normalizing factors in each COUNT(CASE statement, using the largest frequency of all datasets as the numerator, and the frequency for that specific dataset as the denominator. Update the run and project in the WHERE statement.
4. For each tab, copy the revised SQL statement, open the ODBC connection, select all and delete the previous (mangled) SQL and paste in the new.
5. For each normalized tab, select all columns that contain sequence counts (but not distance to V6) and reduce the decimal places to 0
6. For normalized tabs, recalculate each Total column, then reorder the data using Total descending
7. Check totals for Mitch on the TaxByTag-NormVote sheet. Insert a row above the dataset and calculate total number of tags for each dataset.
8. Store a copy of the final spreadsheet(s) in g454/exports/taxonomies.


Interface SecurityConfiguration
-
- All Known Implementing Classes:
AbstractSecurityConfiguration
,DefaultSecurityConfiguration
,SAML2IntSecurityConfiguration
public interface SecurityConfiguration
An interface that enables us to work with security configurations and defaults in a simple fashion.The OpenSAML
ConfigurationService
singleton may be queried for the configuration to use for a certain security operation. For example, to get theEncryptionConfiguration
to use, the following code gives us the config for an encryption operation:EncryptionConfiguration encryptionConfiguration = ConfigurationService.get(EncryptionConfiguration.class);
This is simple and straightforward, and you should probably stick with that way of getting the system defaults for security configuration. However, in some cases, for example when a SAML SP or IdP should support several different profiles regarding security configuration the above doesn't work that well. In these cases you may instantiate different
SecurityConfiguration
objects with different defaults, and use those objects to query for the security configuration.SecurityConfiguration saml2intConfig = setupSaml2intConfig(); ... EncryptionConfig config = saml2intConfig.getEncryptionConfiguration();
When OpenSAML is initialized (using
InitializationService.initialize()
) theConfigurationService
will be assigned the default values from theDefaultSecurityConfigurationBootstrap
class. After OpenSAML has been initialized it is possible to modify these defaults by replacing the stored default objects.EncryptionConfiguration myEncryptionConfiguration = ...; ... a lot of code setting algorithms ... ConfigurationService.register(EncryptionConfiguration.class, myEncryptionConfiguration);
By using a
SecurityConfiguration
object this step may be simplified. For example, to configure the system to use the SAML2Int algorithm requirements you simply do:SecurityConfiguration saml2intConfig = new SAML2IntSecurityConfiguration(); saml2intConfig.initOpenSAML();
If you use the
OpenSAMLInitializer
you can do the following instead:OpenSAMLInitializer.getInstance().initialize( new OpenSAMLSecurityExtensionConfig(), new OpenSAMLSecurityDefaultsConfig(new SAML2IntSecurityConfiguration()));
- Author:
- Martin Lindström (martin@idsec.se), Stefan Santesson<EMAIL_ADDRESS>-
-
Method Summary
All Methods Instance Methods Abstract Methods Modifier and Type Method Description DecryptionConfiguration
getDecryptionConfiguration()
Returns the decryption configuration that has been configured.EncryptionConfiguration
getEncryptionConfiguration()
Returns the encryption configuration that has been configured.String
getProfileName()
Gets the profile name of this configuration setup.SignatureSigningConfiguration
getSignatureSigningConfiguration()
Returns the signing configuration that has been configued.SignatureValidationConfiguration
getSignatureValidationConfiguration()
Returns the signature validation configuration that has been configured.void
initOpenSAML()
Initializes OpenSAML with the defaults that has been installed for this instance.
-
-
-
Method Detail
-
getProfileName
String getProfileName()
Gets the profile name of this configuration setup.- Returns:
- the profile name
-
getEncryptionConfiguration
EncryptionConfiguration getEncryptionConfiguration()
Returns the encryption configuration that has been configured.- Returns:
- encryption configuration
-
getDecryptionConfiguration
DecryptionConfiguration getDecryptionConfiguration()
Returns the decryption configuration that has been configured.- Returns:
- decryption configuration
-
getSignatureSigningConfiguration
SignatureSigningConfiguration getSignatureSigningConfiguration()
Returns the signing configuration that has been configued.- Returns:
- signing configuration
-
getSignatureValidationConfiguration
SignatureValidationConfiguration getSignatureValidationConfiguration()
Returns the signature validation configuration that has been configured.- Returns:
- signature validation configuration
-
initOpenSAML
void initOpenSAML() throws InitializationException
Initializes OpenSAML with the defaults that has been installed for this instance.ConfigurationService.register(XXXConfiguration.class, xxxConfiguration);
- Throws:
InitializationException
- for initialization errors
-
-


Sections deleted from a page will stay with the page. Pages deleted from the content tree are moved to this trash bin.


As a Software Engineer at Google Munich, João has handled various projets in which he worked with different technologies at different levels of the software stack. He's happy to have the chance to talk about his experience at Google in his home university!


Redskins quarterback Dwayne Haskins missed the final play of his first NFL win because he was taking selfies with fans
by Meredith Cash- Washington Redskins quarterback Dwayne Haskins missed the final play of Sunday’s game because he was taking selfies with fans.
- After leading Washington in a gutsy final drive to give his team the 19-16 lead, Haskins headed to the sideline to celebrate.
- The Redskins defence picked off Detroit Lions quarterback Jeff Driskel, forcing the offence to return to the field for one last play.
- When his rookie quarterback was nowhere to be found, interim head coach Bill Callahan sent backup quarterback Case Keenum onto the field to take a knee.
- Visit Business Insider’s homepage for more stories.
Washington Redskins rookie quarterback Dwayne Haskins finally secured his first NFL victory in Week 12 of the season, but he wasn’t able to finish the game out to the final snap.
Fortunately for both Haskins and the Redskins faithful, it wasn’t an injury that kept the former Ohio State great off the gridiron; it was a selfie.
After leading Washington in a gutsy final drive to give his team the 19-16 lead, Haskins headed to the sideline to celebrate.
Then, as the defence took over with just seconds remaining on the clock, the young quarterback headed towards the stands to take photos with fans. But the Redskins picked off Detroit Lions quarterback Jeff Driskel, forcing the offence to return to the field for one last play. When Washington’s coaching staff looked for Haskins to send him in for a victory formation, he was nowhere to be found.
Instead, interim head coach Bill Callahan sent backup quarterback Case Keenum onto the field to take a knee. And even though some coaches likely would have found Haskin’s absence troubling, Callahan spoke about it lightheartedly during the postgame press conference.
“We were looking for [Haskins]. We were looking for him, too,” Callahan said.  I think he thought it was over. So yeah, I’ll have to find out a little bit more.”
For his part, Haskins said he was so excited that he didn’t realise the game was still in play.
“I was so hype, I think I broke a water bottle,” Haskins said. “I look up and we’re in victory [formation]. I thought the game was over with already. But I’ll get it next time.”
Haskins threw for 156 yards on 13 of 29 passing in Sunday’s affair to move the Redskins to 2-9 on the season.
- Read more:
- Russell Wilson and the Seahawks bamboozled the Eagles with a trick play that ended in a glorious 33-yard touchdown
- A Harvard student snuck a brutal hidden message about Jeffrey Epstein into his smack-talk piece published in Yale’s student newspaper
- Magic Johnson lauds LeBron James, says he’s ‘playing as well as I’ve ever seen him play’
- Map shows which states support Colin Kaepernick the most


Bug Fixes
Enhancements
Bug Fixes
Find Dep
s in function calls in the deps()
list.
Dep.putPath
from a variable.
Treat Memoize
defmemo
as def
and defmemop
as defp
.
Resolve exception/1
and message/1
to defexception
.
Resolve to callbacks when searching in any module.
Support arity intervals for unquote_splicing in parameters
Functions defined with unquote_splicing, such as Ecto.Schema.__schema/2
:
for clauses <- Ecto.Schema.__schema__(fields, field_sources, assocs, embeds),
{args, body} <- clauses do
def __schema__(unquote_splicing(args)), do: unquote(body)
end
Need to have their arity not be the number of PsiElements in the parentheses. Any call to unquote_splicing(...)
can end up have 0 to infinite parameters, so it means when one is saw, the range of minimum...maximum
should change to an open interval of minimum...
. This required changing IntRange resolvedFinalArityChange()
to ArityInterval resolvedFinalArityInterval()
on all Call
s, which was a large change. It also meant changing a lot of ArityRange types to ArityInterval, and NameArityRange to NameArityInterval, which influenced the variable names.
Since all Calls support ArityIntervals now and not just special forms and Ecto DSLs, exportArity is changed to always state the ResolveState, so that the special form changes can be integrated for all callers.
The actual implementation of CallImpl.resolvedFinalArityRange is changes to fold over the ArityInterval:
TODO()
true
is
returned now it won't stop the type resolving from working.
OuterElementType
in eex.lexer.TemplateData
.EEX
IElementType
needs to use OuterLanguageElementType
instead of the
direct eex.psi.TokenType
.
QualifiedAlias#text
in runReadAction
for QualifiedAlias#name
null
Chunk
instead of throwing IOException
when read incomplete.beam.FileEditor#getFile
to fix DeprecatedMethodException
as the default
implementation is now deprecated and requires an explicit implementation.
TabbedPaneWrapper.AsJBTabs
instead of JBTabbedPane
for "BEAM Chunks" tabs.JBTabbedPane
stopped showing its labels sometime in the 2020.X IDE version series,
but by debugging when "BEAM Chunks" name was retrieved I found that the bottom tabs used
TabbedPaneWrapper.asJBTabs
. Using that, the labels reappeared.
PsiNamedElement
ElixirAtom
and getting the reference will still work.
Any.isDecompiled
for CallDefinitionHead
.Enhancements
do:
.IndexNotReady
in resolver.Module.multiResolveProject
.__using__
to find quote in any clause. Fixes resolving test macro from
use PowerAssert
quote
's scope when block
injected with
unquote(block)
.field
, timestamps
, and index
in schema
for
use Yacto.Schema
as it makes the block
see the import Yacto.Schema
above
unquote(block)
in the quote
in schema(..., do: block)
.
Enhancements
Reference Resolution
Allow any Alias in a chain to have references.
This allows going to the declaration of Phoenix
, Phoenix.LiveView
, or
Phoenix.LiveView.Socket
depending on whether you're on the Phoenix
,
LiveView
, or Socket
Alias, respectively, in the chain.
Allow any Alias in Qualified Alias to be resolved
This allows going to the declaration of Phoenix, Phoenix.LiveView, or Phoenix.LiveView.Socket
depending on whether you're
on the Phoenix, LiveView, or Socket Alias, respectively, in the chain.
Update DocumentationProvider to work with improved Alias resolution
Without these changes the DocumentationProvider double-resolves and so ends up showing the docs for
def
and defmodule
, instead of the call definition clause or module,
respectively.
Reimplement Module references
Instead of references for only the outermost QualifiableAlias, there is a reference for each right-most
Alias at a given position, so instead of there only be a reference to App.Context.Schema
in
App.Context.Schema
, there is now a reference to App
in the App
prefix, a reference to App.Context
in Context
in App.Context
, and a
reference to App.Context.Schema
in Schema
in App.Context.Schema
.
Not only is this more useful, being able to jump to parent namespaces, but it fixed some of the capability
issues with Go To Definition in the 2020 line of IDEs. This approach of using
getRangeInElement
to target the range of the right-most Alias, while the element was still
the parent that contained, but did not go beyond the Alias, was tried after having references only on
Alias
es and not QualifiedAlias
es did not fix completion issues. It was the
while debugging Go To Declaration actions and noticing they were sensitive to the range in element AND the
docs for PsiReference#getRangeInElement
that I realized that the Go To Declaration and Completion has a hidden requirement that References
for things that behave like namespaces have to work this way.
ModularName
index for Module VariantsisAlias
check.Limit Elixir Module resolution to same JetBrains Project Module
GlobalSearchScope
from allProject
to
moduleWithDependenciesAndLibrariesScope
for faster searching on multi-module projects.
includeTests
based whether the referring element is in a test directory.StubIndex#processElements
instead of home grown forEachNavigationElement
as
processElements
is more efficient.
Favor Module ResolveResults
under same Module
content roots
This should favor deps sources in the same module.
Iterate function
body in unquote(function())
when iterating call definition
clauses in quote
Treats it function
body the same as a __using__
body.
Standardized preferred ResolveResult
filters between Callables and Modules
In Go To Declaration also:
Use the second argument to use
to determine which function is called with apply/3
This pattern is used in Phoenix __using__
, so this lets to differentiate whether
Plug.Conn.assign/3
or Phoenix.LiveSocket.assign/3
is resolved in a Controller,
LiveComponent, or LiveView.
In LEEx Templates
*.html.leex
templates to functions defined in the corresponding
LiveComponent/View module.
assign/3
calls in
update/2
in the LiveComponent or LiveViewTargetElementEvaluator
, so that the
UnqualifiedNoArgumentCall
(name
) that is the identifier in an assign
(@name
) was not counted as valid target for Find Usages / Go To Declaration by itself.
update
to any call definition to cover helper functions and other
callbacks. Don't stop on the first valid match because with helper functions and multiple callbacks,
the last write isn't obvious.
assign/3
assign
calls in |>
pipelinesassign_new/3
@myself
to where it is set in render_pending_components
case
and if
live_component
callslive_component
call itself, so look for any references to the view module
(the LiveComponent) and if it is a live_component
call, then look at those assigns to
try to resolve the assign name.
live_modal
callslive_modal
is not built into phoenix_live_view
, but it is generated by
phx.gen.live
, so most projects will have it. This allows return_to
,
which is used use in live_modal
, to be resolved.
@socket
to last socket variable or call in view module.@flash
to put_flash/3
calls.@inner_content
to Phoenix.LiveView.Utils.to_rendered/2
setting the
assign with put_in
.leex
templates.leex
and use
walking from call definition clauses to any scope
processor lambda, so it can be used to resolve uses of Routes
in .leex
templates.
Resolve Types
@type
, @typep
, or @opaque
;
and named type parameters.
@type
, @typep
, and @opaque
.@callback
as declaring a Typet
for defprotocol
to where it is defined in
Protocol.__protocol__
t
to the same element, so Find Usage finds all
protocol's types instead of just a specific module's type, but this works for projects that have source,
but not yet compiled.
@type t
when available.__MODULE__
Favor source over decompiled per name instead of overall for ResolveResultOrderedSet
This allows the Routes
alias to be resolved to the both the
MyAppWeb.Router.Helpers
in alias MyAppWeb.Router.Helpers, as: Routes
in the
source of MyAppWeb
, but also the decompiled MyAppWeb.Router.Helpers
, which is
necessary as MyAppWeb.Router.Helpers
is produced on compile from router.ex
.
Resolve references through defdelegate
calls.
Resolve code_reloading?
in MyApp.Endpoint
The generated MyApp.Endpoint
for mix phx.new
has a section to enable code-reloading at compile time:
# Code reloading can be explicitly enabled under the
# :code_reloader configuration of your endpoint.
if code_reloading? do
socket("/phoenix/live_reload/socket", Phoenix.LiveReloader.Socket)
plug(Phoenix.LiveReloader)
plug(Phoenix.CodeReloader)
end
Previously, code_reloading?
variable would not resolve because psi.scope.Variable ignored
use
calls, not use
calls are entered and the var!(code_reloading?)
is found in Phoenix.Endpoint.config/1
by way of the unquote(config(opts))
call
in the quote
block in __using__(opts)
.
Resolve qualified calls with unknown resolvable qualifier using only relative identifier and arity.
@spec
@spec
to defdelegate
calls.@specs
to the definitions if the definitions are in a for comprehensionEcto
Query
Resolve reference variables in Ecto.Query
in
passed to
from/2
and the join:
keyword in from/2
.
join/3-5
select/2-3
.where/2,3
binding.a
in [{^assoc, a}]
binding.bindings
in group_by/2-3
.bindings
and expressions
in order_by/2-3
.bindings
in having/2-3
.select_merge
to declaring Ecto.Query
macros.distinct/2-3
as a declaring Ecto.Query
macro.Resolve field
calls in Ecto schema
blocks
How field
works in schema
for Ecto.Schema
use Ecto.Schema
Ecto.Schema.__using__
import Ecto.Schema, only: [schema: 2, embedded_schema: 1]
`Note that only the outer DSLs, schema and embedded_schema are available even though field/2 is defined in Ecto.Schema.
So when you call schema ... do
defmacro schema(source, [do: block])
schema(__CALLER__, source, true, :id, block)
defp schema(caller, source, meta?, type, block)
prelude = quote do
quote blockAt the end of prelude there is
try do
import Ecto.Schema
unquote(block)
after
:ok
end
Hey! An import Ecto.Schema
, but prelude
is just floating as a variable.
At the end of defp schema(caller, source, meta?, type, block)
is
quote do
unquote(prelude)
unquote(postlude)
end
So to statically analyze an Ecto.Schema
module.
schema/2
to defmacro schema
by walking the use
,
__using__
, quote
, and import
.
schema/2
(or macros in general if you want to get fancy 💅 and support
more DSLs),
quote
block at the end (the same as my current __using__
support)
Call
inside an unquote
see if you can resolve it to a
variable in addition to a call definition (which is already supported for Phoenix).
quote
, use the quote block
handling
try
try
and see the import
, walk the import
to find
Ecto.Schema.field/2
API
Ecto.Query.API
functions in Ecto.Query
from(order_by: ...)
from(select: ...)
from(select: tuple())
from(where: ...)
group_by/2-3
having/2-3
select/2-3
expr
argumentwhere
Ecto.Query.API.fragment
to arity interval 0..
.fragment
nested in other Ecto.Query.API
call like
type
.
rightOperand
of join(..., ... in ..., ...)
for
Ecto.Query.API
usages like fragment
.
Resolve module attributes defined in use
__using__
quote
block
org.elixir_lang.reference.resolver.ModuleAttribute
to use
PsiScopeProcessor
for non-@for
and non-@protocol
instead of
custom logic.
use
if the ENTRANCE
is an ancestor since that means the
Alias
on the use
is probably being resolved.
AtUnqualifiedNoParenthesesCall.processDeclarations
will call processor.execute
when it isn't a type spec.
UseScopeSelector
for AtUnqualifiedNoParenthesesCallImpl
has been changed
to SELF_AND_FOLLOWING_SIBLINGS
since the module attribute is used that way. The previous
SELF
value was when the UseScopeSelector
only applied to variables.
"Elixir References" inspection for finding unresolved or invalid references.
visitAtNonNumericOperation
Strip AccessExpression
s from Qualified
qualifiers automatically
Documentation
deprecated
, impl
, and spec
, and actual heads in documentation
for source functions
Documentation
across multiple arities when arity of lookup is ambiguous.
Find variables defined in EEx element
Finds variables defined in EEx PsiElement by creating them the same as a StabBody, which means that the
siblings are checked backwards from the last parent. This fixes resolving the f
from
f = form_for ...
in *input f ...
in LiveView FormComponent templates.
Highlight named parameters in types.
Decompilation
:erlang.beam
:erlang.beam
(even though they aren't actually defined there), there is a shared location for
all reference to point to and then check for Find Usages.
Descend into body of unless
in quote
for treeWalkUp
.
Descend into for
when looking for call definition clauses.
QuoteMacro.treeWalkUp
into if
unless
was already walked, but not if
. Fixes resolving Repo.preload/2
,
which is defined inside an if
in use Ecto.Repo
's __using__
.
Add defguard
and defguardp
to CallDefinitionClause.is
.
Don't generate a reference for Elixir
Elixir
is not declared anywhere. It is the namespace of all Aliases.
Refactor how sibling sequences are walked
Port filtering out comments, white space and end of expressions from siblingExpression
.
Reimplement fullyQualifiedName
Eliminate the specialization for ElixirAlias
AND QualifiableAlias
as which was
picked as based on the static casted type. fullyQualifiedName
will include the following:
__MODULE__
to the enclosing modular canonical nameElixirMultipleAliases
will combine the qualifier (qualifier.{...}
)
outside the {}
with the relative name inside ...{relative, ...}
.
EEx
viewFile
for .eex
files used in same directory as .ex
using
EEx.function_from_file
.
EEx.function_from_file/3-5
.templates
directory.Walk quote
macros for sibling call definitions
Always fallback to name/arity in any module if callable not resolved in scope
Resolve uses of aliased name produced by alias __MODULE__
ParsingTestCase
to latest APIs.runPluginVerifier
Bug Fixes
Fix Ctrl+Click/Cmd+B/Go To Declaration for Aliases.
Don't index non-canonical names.
The non-canonical name for a nested defmodule
is the Alias given to that defmodule
call. While that name should be resolvable within the parent modular, it should not be indexed because the index is for the global scope.
Prevents defmodule Enum
in
defmodule ExJsonSchema.Validator.Error do
...
defmodule Enum do
...
end
...
end
from being indexed as Enum
, which hides the decompiled version from the standard library because
source references are favored over decompiled versions when resolving references.
Reference Resolution
Only favor same file Module resolutions if they are valid
Invalid same file results can occur for prefix matches, but if only prefix matches are available, then a
whole project search should occur instead of only when there are no same file ResolveResult
of either validity.
Include library sources in Module#multiResolveProject
GlobalSearchScope
Ensures that library sources can be returned, which are favored over decompiled libraries, so that
__using__
macros can be followed.
Protect from Module
reference element not having a VirtualFile
when checking if
a test file.
Fix QualifiableAliasImpl#fullyQualifiedName
MultipleAliases
alias
line like alias Myapp.MyContext.{Schema1, Schema2}
,
Schema1
would think its fullyQualifiedName
was only Schema1
, when
it should included the fullyQualifiedName
of the qualifier, MyApp.MyContext
too. This leads to the correct Schema1
fully-qualified name of
MyApp.MyContext.Schema1
. This fix makes references to Schema1
resolve to
both the alias
and the defmodule
.
MultipleAliases
for deep aliasesalias
line like
alias Myapp.MyContext.{NestedContext1.Schema1, NestedContext2.Schema2}
,
Schema1
would think its fullyQualifiedName
was only
NestedContext1.Schema1
, when it should included the fullyQualifiedName
of the
qualifier, MyApp.MyContext
too. This leads to the correct Schema1
fully-qualified name of MyApp.MyContext.NestedContext.Schema1
. This fix makes
references to Schema1
resolve to both the alias
and the
defmodule
.
In LEEx Templates
false
, true
, nil
)|> case do ... end
None
calls were ignored, but when case
is used in a pipeline it is a
None
since it has no literal arguments and only resolved arguments, so it was being
ignored.
if
sstab
of the doBlock
, but the else
is in the
blockList
.
Don't use the same path for actual aliases and defmodules when resolving aliases
Prevents a case of an exact match and its namespace both being marked as valid (exact matches).
Types
required/1
and optional/1
)
...
in lists in type specs.ElixirStabBody
for ancestorTypeSpec
.Callable
reference for unquote
in typespec instead of
Type
reference.
Allow modular names to resolve to multiple modulars
Fix :systemd.ready()
not resolving because :systemd
only exists as a decompiled
file from source because it is Erlang and there is no preference given to different MIX_ENVs. All
MIX_ENV defmodule :systemd
are returned for the :systemd
and all of them are
checked for ready/0
.
Fix resolving qualified calls that are defined through use
Resolving qualified calls only used Modular.callDefinitionClauseCallFoldWhile
, which ignores
all non call definition, which means it ignored the use
, but switching to use
org.elixir_lang.psi.scope.call_definition_clause.MultiResolve
PsiScopeProcessor
used for unqualified calls, but starting on the modular call of the qualifier, all the use
handling that works for unqualified calls also works for qualified calls now.
This fixes resolving MyAppWeb.Endpoint.config_change(changed, removed)
in
MyApp.Application.config_change/3
because
use Phoenix.Endpoint, otp_app: :my_app
in MyAppWeb.Endpoint
is now walked for
qualified calls.
Fix resolving variables to parameters that are both pattern matched and defaulted
In functions like
def f(%Struct{} = s \\ %Struct{}) do
s
end
s
in the body would not resolve to the s
parameter because the default
parameter had to be PsiNamedElement
, that is a variable directly. Instead, recurse on any
operand before the \\
to support pattern matching too.
Use keepProcessing()
in MultiResolve.addToResolveResults
instead of specific
return values
Fixes def
in def default_host_flag, do: "--host
resolving as an invalid match to
default_host_flag
instead of search the implicit imports for defmacro def
.
The nameIdentifier
for a defdelegate
is the first argument, not the
as:
The as:
is the name in the to:
module only.
Fix defdelegate
MultiResolve
as
name for checking for a name matchdefdelegate
as matching if the as
name as found in
to
module, but if the head alone is prefixed by the name then the head should be a
ResolveResult element even if the to
or as
can't be found
to
, I only checked if the children were call definition clauses, but this meant
that all the for
handling was ignored, so start a new resolveResults()
for
each modular.
Fix toModulars
not returning all modulars
fullyResolve
only touched the first modular.
Favor complete valid results over incomplete invalid earlier bindings for variables.
Ecto
assoc/2
pseudo-function in
join(..., _ in assoc(_, _))
Fix unaliasing multiple aliases
UnaliasName
still assumed that fullyQualifiedName
included the qualifier, but it
is just the lexical qualifier now.
Check for MULTIPLE_ALIASES_QUALIFIER
in ResolveState
for prefix matches.
Fixes resolving SSH.Key
to Foo.SSH.Key
when the alias is
alias Foo.{SSH, ...}
.
Ignore type variables in type restrictions when resolving normal variables.
Fix withSelf
for childExpressions
siblingExpressions
call
While refactoring the walk, I copied the call for siblings, which shouldn't included self. Children
should include self, or the first expression is missed which showed up as Repo.transaction
not being found because transaction
is the first def
inside an if
in Ecto.Repo.__using__
's quote
block.
Don't try to resolve keys/fields of a capture.
Don't generate reference for assoc
in *join*: .. in assoc(..., ...)
.
Already worked for join/3
calls, extend to keyword syntax too.
Fix resolving references to Modulars
when resolved is in BEAM file
The check to see if the resolved was a modular only worked for source elements because it checked if it
was a Call
, but decompiled modules in BEAM files are ModuleImpls
.
Fix logic error when ignored pinned variables as declaration sites
The logic was supposed to be that
... but, the code was operatorText != "^" && execute(match as Call, state)
which
meant that if the operatorText
was "^"
, then the whole expression was
false
, which meant keepProcessing
was false
, so any pin stopped the
processing for other unpinned parameters after the pin in addition to processing in the outer scope of the
pin.
Prefer source only for target candidates for Go To Declaration and not all ResolveResults.
Ensures that functions only defined in decompiled Modules can be found because the Module's ResolveResults
include the decompiled Module in addition to the source ones.
Don't prefer source over decompiled in ResolveResultOrderedSet
Preference is now handled in Resolver
, so that source preference only happens on
Go To Declaration and not for multiResolve
because that needs decompiled Modulars for
compile-time for
loops that are too complex to infer.
Use primary arity instead of final for resolving Callables
Fixes unquote
in def unquote(...)() do
not resolving.
Filter out decompiled private functions from completions
If they are only available as decompiled functions, it is unlikely we have the source available to make them
public, which was the reasoning to allow private functions at all.
Decompiling
and
when not and/2:hipe_arm_encode
.unquote
false
when decompiling.:thrift_json_parser
. Add true
to the Unquote
list
too since it would have the same issue.
in
when not in/2
:digraph_utils
containingFile.originalFile
for resolve.Module.multiResolveProject
VirtualFile
is available for decompiled files.Performance
Don't use .navigationElement
for Variants
Using navigationElement
forces parsing of the decompiled source, which isn't necessary whe
only showing the lookup string.
Remove ModuleOwner
interface
BeamFileImpl
is known to have only 1 module in it, so it makes the code more complicated to
make it act like source files.
getChildrenByType
, as it causes the decompiled source to be parsed, which is slow
and unnecessary since all the metadata for modules is available from the binary format.
Get project from parent in decompiled Module and CallDefinition
Not doing this causes an unnecessary parsing of the decompiled source.
Fix infinite recursion in ElixirVisitor#visitLiteralSigilLine
Show more scope for keyword key Go To Definition
live_modal
may be used more than once in the same file.
Put entrance
as initial visited in module.Variants
.
Check if PsiElement
has been visited before walking quote
and __using__
Don't mark qualified calls as unresolved in "Elixir References" inspection
I don't have a good approach for resolving struct fields or map keys for now, don't mark them as unresolved as
it clutters finding unresolved calls I expect to work.
Don't assume PsiFile#virtualFile
is NotNull
in
PsiElement.moduleWithDependentsScope
.
Don't assume PsiFile#virtualFile
is NotNull
in
AtNonNumericOperation.computeReference()
.
putInitialVisitedElement
in variable.MultiResolve
Fixes resolving variables through use
statements for variables in the body of
defmodule
blocks.
Log when VISITED_ELEMENT_SET
is null
instead of erroring.
Fix resolvePrimaryArity
for |> case do
resolvedPrimaryArity
did not add both the doBlock's arity and the arity from the
|>
at the same time. If the pipe was there, the do
was ignored because the
pre-existing resolvePrimaryArity
was not used if the normal primary arity of the call was zero,
which is the case for |> case do
.
Find module for elements in libraries
Elements in libraries don't have a Module
from ModuleUtil.findModuleForPsiElement
,
so scan the element's project's module's content roots for the closest Module
for libraries.
prependQualifiers
when typing an alias on a new line in the body of a function.
Don't descend into quote
blocks if the call being resolved is given to an unquote
in
the quote
block.
updateSinceUntilBuild
to prevent breaking in future IDEs.findModuleForPsiElement
.BasePlatformTestCase
instead of LightCodeInsightTestCase
because LightCodeInsightTestCase
is deprecated.


If the embedded video does not work, you can view the video here.
{% else %}{{ include.page.summary }}
{% if include.page.award %}{{ include.page.award }}
{% endif %} {% if include.page.website %}Read more at {{ include.page.website }}.
{% endif %} {% if include.page.references == true %}The project uses the {{ include.page.refitem }} framework and is created by {{ page.refauthor }}.
{% endif %}


--- layout: page permalink: "/community-commissioned/artscape/" ---
This organisation provides a wide range of creative projects, for the benefit of people who use the services of the Oxfordshire and Buckinghamshire Mental Health Partnership NHS Trust. The images are a sample of teenagers work during a range of projects that explored feelings, places, and spaces created and lost. The new Highfield Unit, Warnford Hospital (opened Autumn 2012) adopted 20 of their images to hang in a variety of their rooms. (Example images 1.2.3)


MoinWorld
Mentor, former marketing intern and active member of the MoinWorld community, the Hamburg branch of Women Techmakers. A community offering coding courses, meet-ups, mentorship for women and girls.
Hamburg Dockers
Huge fan and member of the Hamburg Dockers, an Australian Football team.
Literature
Avid reader of English and German novels. Always on the look for the next book that I can't put down.


requirement: you must have pgadmin installed for this review.
This document is meant to serve as a checklist review of materials taught in the Node Unit of the Full Stack Web Curriculum at Lambda School.
Upon completion of this review, you should have a simple application that could be used as a boilerplate, upon which you can build out custom models, routes, and helpers to suit individual needs. However, don’t use a boilerplate to do something like this until you can build it from scratch on your own. Do the reps to earn it, this document is just a guide.
The code samples included in this review should be considered a ‘bare minimum’ and should be embellished on in any way. In some instances there is no code included, simply instructions to author something on your own. Seriously don’t just copy this. Write excellent tests, thoughtful documentation, and return meaningful data. Some commands and code examples may differ slightly from what you were taught in lectures, but the underlying principles will remain the same.
Create a directory for your API to live in
In your terminal:
mkdir < server-directory-name > && cd < server-directory-name >
NOTE: the
&&
in this first command executes both commands in sequence, or one after the other. Having this command on one line creates the directory and then changes into it so that the following commands are executed in the newly created root directory. Great for this use case, but use with caution in other contexts.All further terminal commands in this review take place in the root directory.
git init
npx gitignore node
//creates .gitignore for node
In your terminal:
git add .gitignore
git commit -m “Initial Commit”
git remote add origin <remote_repo_rl>
(the url from your github remote)
git push -u origin master
git checkout -b “initialize”
git push -u origin initialize
Now you have a remote repo connected to your local repo, and you’re currently developing on a newly published branch called initialize.
In your terminal:
npm init -y
//creates package.json
git add package.json
git commit -m “Initializes Node/NPM”
NOTE: The
-y
flag answers ‘yes’ to allnpm init
configuration questions. We don’t need anything more configured than the stock package.json for now, but explorenpm init
without the-y
flag later so you have a better idea of what’s going on here.
In your terminal:
npm i express helmet cors knex knex-cleaner dotenv pg bcryptjs jsonwebtoken
npm i nodemon cross-env jest supertest -D
git add package.json package-lock.json
git commit -m “Installs dependencies”
package.json
The start
script here is used by Heroku. The value we give it tells Heroku to run node index.js
to start our server.
In your terminal:
git add package.json
git commit -m “Updates package.json”
git push
NOTE: You may notice that the
-m
message in each of these commits so far is taken from the header of that section. Throughout this review, continue to follow the commit format laid out above, and if you need to see which files toadd
usegit status
. Following this commit flow will yield a really nice commit history that can also serve as a checklist on future reps through this development process. GIT HYGIENE IS SO IMPORTANT TO GOOD EMPLOYERS!!!!
Start a new branch called tests/server-endpoint
:
git checkout -b “tests/server-endpoint”
git push -u origin tests/server-endpoint
Add the following files to the root directory, and don’t forget the API
directory:
[ ] API/server.spec.js
[ ] build your application’s entry point in index.js
index.js
const server = require('./API/server.js');
const port = process.env.PORT || 5000;
server.listen(port, () => {
console.log(`\n<<< Server running on port ${port} >>>\n`);
})
API/server.spec.js
const request = require("supertest");
const server = require("./server.js");
it("should set db environment to testing", function() {
expect(process.env.DB_ENV).toBe("testing");
});
describe("server", function() {
describe("GET /", function() {
it("should return 200", function() {
// run the server
// make a GET request to /
// see that the http code of response is 200
return request(server)
.get("/")
.then(res => {
expect(res.status).toBe(200);
});
});
it("should return HTML", function() {
return request(server)
.get("/")
.then(res => {
expect(res.type).toMatch(/html/i);
});
});
});
});
package.json
"scripts": {
"server": "nodemon",
"start": "node index.js",
"test": "cross-env DB_ENV=testing jest --watch"
},
API/server.js
Then run our tests to watch them fail first,
in your terminal:
[ ] npm run test
[ ] Now commit this work and push it to the remote.
[ ] Create a new branch called feat/server-endpoint
and push it up to the remote.
[ ] Let’s finish building out our basic server endpoint so that test can pass:
API/server.js
const express = require('express');
const cors = require('cors');
const helmet = require('helmet');
const server = express();
server.use(helmet());
server.use(cors());
server.use(express.json());
server.get('/', (req, res) => {
res.send('
🚀
');
})
module.exports = server;
Then, in your terminal:
npm run server
, and once you see the server is running in your console,
[ ] npm run test
(in a new terminal window, or stop the server first)
[ ] Commit this work, as long as your test passes.
Adding a logger middleware early is a good idea, so let’s do that now.
[ ] Make a new branch called middleware/logger
and push it up to the remote.
middleware
[ ] Create a file called logger.js
inside of middleware
middleware/logger.js
module.exports = logger;
function logger(req, res, next) {
console.log(`${req.method} to ${req.url} at ${new Date().toISOString()}`)
next();
};
.use
the logger in our server.js
API/server.js
const express = require('express');
const cors = require('cors');
const helmet = require('helmet');
const logger = require('../middleware/logger');
const server = express();
server.use(helmet());
server.use(cors());
server.use(express.json());
server.use(logger);
server.get('/', (req, res) => {
res.send('
🚀
');
})
module.exports = server;
[ ] Run your server and point your browser to localhost:5000
. You should see the rocket emoji in your browser window, and when you come back to your terminal you should see something like: GET to / at dateTime
, which means your logger is logging.
[ ] Commit this work and push it up.
We now have a super simple express server running, so it’s time to deploy! Try to deploy as soon as you have something working, so that whenever you’re ironing out any issues in staging, you’ll only have to troubleshoot the smallest codebase possible. Deploy early, commit often!
To explain the 3 different environments we’ll be using:
master
master
once staging looks good
[ ] Signup/login to a free Heroku account.
[ ] Once logged into your Heroku dashboard, click on New
in the top Right corner, and then on Create new app
[ ] Give your app a meaningful name, and be sure to use the word staging
to differentiate it from the production
app we’ll build in a bit. Then click on Create app
[ ] Once your app is created, navigate to the Deploy
tab (Heroku tends to drop you off here anyhow) and select GitHub
as the Deployment Method.
[ ] Connect the app to your remote GitHub respoitory, and enable automatic deploys on master
branch.
Now, since our master
branch currently only contains a .gitignore file, we’ll need to merge our latest middleware/logger
branch into master
so Heroku will have something to deploy.
In your terminal:
git checkout master
since we know there are no changes on the remote that we need to pull into our branch first, we can:
git merge middleware/logger
git push origin master
If there were remote changes that we needed to pull into our local branch first, we would need to instead run:
git checkout master
git pull –rebase origin master
git checkout middleware/logger
git pull –rebase origin master
git checkout master
git merge middleware/logger
git push origin master
Now in your Heroku Dashboard, you can click on the Activity
tab and watch Heroku build the application.
[ ] Once you see a Build Succeeded
and a Deployed
message in the Activity
tab, click on ‘Open app’ in the top right corner and hope for a rocket!
[ ] While you’re in your Heroku dashboard, go ahead and create a new app for production following the above guidance. Connect it to the exact same repository’s master
branch just DON’T deploy it yet!
Now that we have a staging app deployed, it’s time to create and update our README.md to reflect the changes we’re about to take live on our production deployment. You’ll want to follow this deployment flow to allow for the smoothest introduction of new features for anyone working on a client that would be consuming this API: > deploy to staging > test/troubleshoot staging deploy > update docs > deploy to production
If you follow this flow every time you’re deploying new features, you’ll save yourself the rush to explain everything you’ve changed. All the new features will be documented in the master
README before the production
deploy completes, so your team will remain informed about the current state of the deploy.
Use the Standard Readme spec for guidance, and include plenty of meaningful info in your README like a link to the deployed production API (if you already created your production deploy in Heroku, the URL for the production app will be https://NAME-OF-PRODUCTION-APP-HERE.herokuapp.com
)
docs/README
and push it up to the remote. Then merge that branch into master
Now that we’ve updated our docs and been through any troubleshooting on the staging app, we can deploy to production.

Your staging app is continuously deploying from master
because we want our newest edits to be live for us to test asap, but we manually deploy production so that we can be sure everything is working correctly before we take our new edits live.
Next we’ll create our database and add routes for users to register and login.


My Fitness Website
Home
Workout
Muscle Group Selector
shoulder
chest
abs
biceps
Triceps
Glutes
Quadriceps
Hamstring
Calves
Trapezius
Upper Back
Lower Back
Calorie Calculator
Create a Meal Plan
Sign Up
Login
Search
Click a muscle to see workouts
Front
Back


You are not allowed to open the requested document. One of the following may have occurred:
- You have already claimed two documents in this status. Please release one of the documents if you would like to open another.
- The document you are trying to open has already been claimed.
- You have already submitted work for this document.
- Your login has expired.
Please close this window and refresh the file listing page.


Menu
Dashboard
Data
Datatable
Map
IRIS Vaccine Tracker
COVID-19
Dashboard
Map
IRIS Vaccine Tracker
COVID-19
Map
Vaccination Heat Map
Total Vaccinations
Percentage


{% extends 'panel.html' %} {% load utilities %} {% block head_title %}Bad token{% endblock %} {% block panel_content %}
The password reset link was invalid, possibly because it has already been used. Please request a new password reset.


Deterministic Optimization
Computational Data Analytics / Machine Learning
Financial Modeling
Data and Visual and Analytics
Regression Analysis
Data Analytics in Business
Collaborated across teams and team members in an agile environment to investigate and solve problems on newly released software
Used tools such as Jira and Salesforce to assist in deployment of release fixes
Troubleshoot business intelligence dashboards through virtual environments, fixing APIs calls, Ports, Firewalls, SQL scripts, server logs, SaaS software yaml files
Directly customer facing. Needed to be able to solve difficult technical situations under pressure as the customer could be stressed
Increasing company efficiency through programmatically solving issues
Building models(neural network, regression analysis etc), selecting features, and optimizing classifiers using machine learning techniques to accurately predict drug efficacy
Data mine to create robust database for ad-hoc analysis of drug development
Lead Data Science Department meetings
Project proposal planning,writing, editing and submission for new projects to increase portfolio of company as a whole and progress in the cure for Parkinson’s Disease
Simple stochastic models to price derivative securities in various asset classes including equities, fixed income, credit and mortgage-backed securities
Develop derivatives pricing models
Asset allocation and portfolio optimization
Real options, commodity and energy derivatives and algorithmic trading
1 - Basic 2 - Novice 3 - Intermediate
4 - Advanced 5 - Expert
{{ item[0] }} - {{ item[1] }}


Agenda
- What is Virtual Reality?
- Why WEB VR?
- Tech stack
- Demo
- Opportunities for us
What is VR?
Virtual reality (VR) is an interactive computer-generated experience taking place within a simulated environment.
Why WEB VR?
- We are web developers
- We already know it
- We know how to support it
- We know how to implement it
- Everything works in browser
Tech stack
- Three.js
- React360
- A-frame
Tech stack
Three.js
React360
- A-frame
How we can use it
-
Education apps
- Biology
- Сhemistry
- Physics
- History
- Skills training
- Distance learning
- And many others...