Adding Glottocodes to database

Adding GlottoCodes to database, linking to societies and trees.
You need to run load_all_datasets.sh to get the new data for this pull request.
Still needs a bit of work (mostly to prune the trees properly) and to classify the languages.

Hi Stephanie,
I pulled the new stuff on my local machine.
But load_all_datasets.sh failed with lots of errors. Here the short version. Maybe you can see the point.
(env_dplace)dplace_user@precise64:/var/django/dplace$ ./load_all_datasets.sh
Loading ISO Codes from Ethnologue
ISO Code 'areg' too long, skipping
Loading ISO Code Lat/Long
Tried to attach Lat/Long to ISO Code drw but code not found
Tried to attach Lat/Long to ISO Code sgl but code not found
Tried to attach Lat/Long to ISO Code tnf but code not found
Tried to attach Lat/Long to ISO Code bjd but code not found
Tried to attach Lat/Long to ISO Code ppv but code not found
Tried to attach Lat/Long to ISO Code tpw but code not found
Tried to attach Lat/Long to ISO Code xyl but code not found
Tried to attach Lat/Long to ISO Code btb but code not found
Tried to attach Lat/Long to ISO Code ayx but code not found
Skipped ISO Code mja -- ignored
Tried to attach Lat/Long to ISO Code mja but code not found
Tried to attach Lat/Long to ISO Code pcr but code not found
Tried to attach Lat/Long to ISO Code xia but code not found
Tried to attach Lat/Long to ISO Code myq but code not found
Tried to attach Lat/Long to ISO Code gft but code not found
Tried to attach Lat/Long to ISO Code gmo but code not found
Tried to attach Lat/Long to ISO Code cnm but code not found
Tried to attach Lat/Long to ISO Code ixj but code not found
Tried to attach Lat/Long to ISO Code jai but code not found
Tried to attach Lat/Long to ISO Code ckk but code not found
Tried to attach Lat/Long to ISO Code cke but code not found
Tried to attach Lat/Long to ISO Code ckc but code not found
Tried to attach Lat/Long to ISO Code cki but code not found
Tried to attach Lat/Long to ISO Code ckj but code not found
Tried to attach Lat/Long to ISO Code ckd but code not found
Tried to attach Lat/Long to ISO Code ckf but code not found
Tried to attach Lat/Long to ISO Code ckw but code not found
Tried to attach Lat/Long to ISO Code cbm but code not found
Tried to attach Lat/Long to ISO Code cun but code not found
Tried to attach Lat/Long to ISO Code quu but code not found
Tried to attach Lat/Long to ISO Code quj but code not found
Tried to attach Lat/Long to ISO Code qxi but code not found
Tried to attach Lat/Long to ISO Code qut but code not found
Tried to attach Lat/Long to ISO Code mvc but code not found
Tried to attach Lat/Long to ISO Code mms but code not found
Tried to attach Lat/Long to ISO Code mpf but code not found
Tried to attach Lat/Long to ISO Code mvj but code not found
Tried to attach Lat/Long to ISO Code poa but code not found
Tried to attach Lat/Long to ISO Code pou but code not found
Tried to attach Lat/Long to ISO Code pob but code not found
Tried to attach Lat/Long to ISO Code mtz but code not found
Tried to attach Lat/Long to ISO Code tzt but code not found
Skipped ISO Code aay -- ignored
Tried to attach Lat/Long to ISO Code aay but code not found
Tried to attach Lat/Long to ISO Code dha but code not found
Tried to attach Lat/Long to ISO Code dap but code not found
Tried to attach Lat/Long to ISO Code elp but code not found
Tried to attach Lat/Long to ISO Code hrr but code not found
Tried to attach Lat/Long to ISO Code ilw but code not found
Tried to attach Lat/Long to ISO Code tlz but code not found
Tried to attach Lat/Long to ISO Code tlw but code not found
Tried to attach Lat/Long to ISO Code eml but code not found
Tried to attach Lat/Long to ISO Code dkl but code not found
Tried to attach Lat/Long to ISO Code dwl but code not found
Tried to attach Lat/Long to ISO Code cti but code not found
Tried to attach Lat/Long to ISO Code hva but code not found
Tried to attach Lat/Long to ISO Code hsf but code not found
Tried to attach Lat/Long to ISO Code yus but code not found
Tried to attach Lat/Long to ISO Code nln but code not found
Tried to attach Lat/Long to ISO Code tzb but code not found
Tried to attach Lat/Long to ISO Code tzc but code not found
Tried to attach Lat/Long to ISO Code tze but code not found
Tried to attach Lat/Long to ISO Code tzu but code not found
Tried to attach Lat/Long to ISO Code tzs but code not found
Tried to attach Lat/Long to ISO Code tzz but code not found
Tried to attach Lat/Long to ISO Code drh but code not found
Tried to attach Lat/Long to ISO Code ccq but code not found
Tried to attach Lat/Long to ISO Code cka but code not found
Tried to attach Lat/Long to ISO Code pbz but code not found
Tried to attach Lat/Long to ISO Code sca but code not found
Tried to attach Lat/Long to ISO Code ybd but code not found
Tried to attach Lat/Long to ISO Code yos but code not found
Tried to attach Lat/Long to ISO Code pgy but code not found
Tried to attach Lat/Long to ISO Code yma but code not found
Tried to attach Lat/Long to ISO Code meg but code not found
Tried to attach Lat/Long to ISO Code sum but code not found
Tried to attach Lat/Long to ISO Code auo but code not found
Tried to attach Lat/Long to ISO Code ibi but code not found
Tried to attach Lat/Long to ISO Code izi but code not found
Tried to attach Lat/Long to ISO Code jar but code not found
Tried to attach Lat/Long to ISO Code nfg but code not found
Tried to attach Lat/Long to ISO Code nfk but code not found
Tried to attach Lat/Long to ISO Code twc but code not found
Tried to attach Lat/Long to ISO Code gav but code not found
Tried to attach Lat/Long to ISO Code grz but code not found
Tried to attach Lat/Long to ISO Code sic but code not found
Tried to attach Lat/Long to ISO Code mgx but code not found
Tried to attach Lat/Long to ISO Code tgg but code not found
Tried to attach Lat/Long to ISO Code bhk but code not found
Tried to attach Lat/Long to ISO Code bkb but code not found
Tried to attach Lat/Long to ISO Code kgh but code not found
Tried to attach Lat/Long to ISO Code mst but code not found
Tried to attach Lat/Long to ISO Code myt but code not found
Tried to attach Lat/Long to ISO Code kzh but code not found
Tried to attach Lat/Long to ISO Code tie but code not found
Tried to attach Lat/Long to ISO Code ghc but code not found
Tried to attach Lat/Long to ISO Code okl but code not found
Tried to attach Lat/Long to ISO Code aqp but code not found
Tried to attach Lat/Long to ISO Code bll but code not found
Tried to attach Lat/Long to ISO Code crr but code not found
Tried to attach Lat/Long to ISO Code chs but code not found
Tried to attach Lat/Long to ISO Code krb but code not found
Tried to attach Lat/Long to ISO Code mjy but code not found
Tried to attach Lat/Long to ISO Code mkq but code not found
Tried to attach Lat/Long to ISO Code mof but code not found
Tried to attach Lat/Long to ISO Code nwa but code not found
Tried to attach Lat/Long to ISO Code ofo but code not found
Skipped ISO Code pie -- ignored
Tried to attach Lat/Long to ISO Code pie but code not found
Tried to attach Lat/Long to ISO Code tta but code not found
Tried to attach Lat/Long to ISO Code mzg but code not found
Loading Glottolog Languages
Traceback (most recent call last):
File "./dplace_app/load.py", line 88, in
run(sys.argv[1], sys.argv[2])
File "./dplace_app/load.py", line 37, in run
load_glottocode(dict_row)
File "/var/django/dplace/dplace_app/load/glottocode.py", line 16, in load_glottocode
glotto, created = GlottoCode.objects.get_or_create(glotto_code=glotto_code)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/manager.py", line 154, in get_or_create
return self.get_queryset().get_or_create(**kwargs)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 376, in get_or_create
return self.get(**lookup), False
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 304, in get
num = len(clone)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in len
self._fetch_all()
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: relation "dplace_app_glottocode" does not exist
LINE 1: ...."id", "dplace_app_glottocode"."glotto_code" FROM "dplace_ap...
^
Loading Languages from Ethnologue
Traceback (most recent call last):
File "./dplace_app/load.py", line 88, in
run(sys.argv[1], sys.argv[2])
File "./dplace_app/load.py", line 53, in run
load_lang(dict_row)
File "/var/django/dplace/dplace_app/load/language.py", line 35, in load_lang
language = Language.objects.get(iso_code=isocode)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 304, in get
num = len(clone)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in len
self._fetch_all()
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column dplace_app_language.glotto_code_id does not exist
LINE 1: ...age"."name", "dplace_app_language"."iso_code_id", "dplace_ap...
^
Traceback (most recent call last):
File "./dplace_app/load.py", line 88, in
run(sys.argv[1], sys.argv[2])
File "./dplace_app/load.py", line 53, in run
load_lang(dict_row)
File "/var/django/dplace/dplace_app/load/language.py", line 35, in load_lang
language = Language.objects.get(iso_code=isocode)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 304, in get
num = len(clone)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in len
self._fetch_all()
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column dplace_app_language.glotto_code_id does not exist
LINE 1: ...age"."name", "dplace_app_language"."iso_code_id", "dplace_ap...
^
... and so forth
Loading EA Variables
Loading EA Variable Codes
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 4307,,,,,,,,,,,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5048,,,92,N,CODE,N,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5061,,,1,,a ,43,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5062,,,1,,a ,13,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5063,,,1,,a ,18,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5064,,,1,,a ,45,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5065,,,1,,a ,17,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5068,,,1,,b ,29,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5069,,,1,,b ,8,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5070,,,1,,b ,9,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5071,,,1,,b ,39,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5072,,,1,,b ,9,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5075,,,1,,c ,20,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5076,,,1,,c ,11,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5077,,,1,,c ,13,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5078,,,1,,c ,34,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5079,,,1,,c ,18,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5087,,,1,,d ,21,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5088,,,1,,d ,15,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5089,,,1,,d ,13,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5090,,,1,,d ,67,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5091,,,1,,d ,9,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5099,,,1, ,e ,8,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5100,,,1,,e ,8,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5101,,,1, ,e ,39,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5102,,,1, ,e ,21,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5103,,,1,,e ,12,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5111,,,1, ,f ,5,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5112,,,1,,f ,11,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5113,,,1, ,f ,17,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5114,,,1, ,f ,15,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5115,,,1,,f ,9,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5123,,,1, ,g ,5,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5124,,,1,,g ,14,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5125,,,1, ,g ,21,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5126,,,1, ,g ,15,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5127,,,1,,g ,5,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5135,,,1, ,h ,11,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5136,,,1,,h ,10,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5137,,,1, ,h ,14,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5138,,,1, ,h ,27,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5139,,,1,,h ,9,,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5147,,,1, ,i ,12,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5148,,,1,,i ,20,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5149,,,1, ,i ,14,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5150,,,1, ,i ,9,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5151,,,1,,i ,10,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5159,,,1, ,j ,10,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5160,,,1,,j ,16,,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5161,,,1, ,j ,10,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5162,,,1, ,j ,14,,,,,,,,,,,,,,
load_ea_codes: did not get anything from ./datasets/csv/ea_codes.csv row 5163,,,1,,j ,11,,,,,,,,,,,,,,,
Loading EA Variable Values
Traceback (most recent call last):
File "./dplace_app/load.py", line 88, in
run(sys.argv[1], sys.argv[2])
File "./dplace_app/load.py", line 51, in run
load_ea_val(dict_row)
File "/var/django/dplace/dplace_app/load/society_ea.py", line 251, in load_ea_val
society = Society.objects.get(ext_id=ext_id)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 304, in get
num = len(clone)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in len
self._fetch_all()
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column dplace_app_society.xd_id does not exist
LINE 1: ...app_society"."id", "dplace_app_society"."ext_id", "dplace_ap...
^
Loading Binford Variables
Original latitude (published in book)
Original longitude (published in book)
Revised by Carlos Botero, if original lat/long not on land or clearly misplaced
Revised by Carlos Botero, if original lat/long not on land or clearly misplaced
Year of ethnographic reporting (ENVCALC2)
Subsistence beyond hunting-gathering-fishing at time of documentation?
Total number of persons to whom the the ethnographic description applies
Ethnographers' estimates of the total land area occupied by the group, in units of 100 square kilometers
Population density (Total population/Area)
Gathering: percent dependence upon terrestrial plants
Hunting: percent dependence on game
Fishing: percent dependence upon aquatic organisms
Food type that provides the majority of a group's nutritional intake
Mobility of group
Size of smallest group that regularly cooperates for subsistence - version in Table 5.01
Size of group that camps together when population is most aggregated (for non-sedentary groups) or village size (for sedentary groups) - version in Table 5.01
Size of periodic regional aggregations - version in Table 5.01
Number of moves per year - version in table 5.01
Distance moved per year (miles) - version in table 5.01
Stature (cm) - Male
Stature (cm) - Female
Height qualifier
Weight (kg) - Male
Weight (kg) - Female
Weight qualifier
Size of smallest group that regularly cooperates for subsistence - version in Table 8.01
Size of group that camps together when population is most aggregated (for non-sedentary groups) or village size (for sedentary groups) - version in Table 8.01
Size of periodic regional aggregations - version in Table 8.01
Number of moves per year - average of values presented in table 8.04
Distance moved per year (miles)- average of values presented in table 8.04
Percentage polygyny (Percentage of males with multiple wives)
Qualifier for percentage of polygyny (> or <)
Mean age of males at first marriage.
Mean age of females at first marriage.
Persons per household
Household made up  of two nuclear families
Presence of communal structure
Persons per family
Household type
System states (Groupings of foragers; e.g., mounted hunters, agriculturalists, mutualists, etc.)
Number of households per GROUP1 unit
Number of households per GROUP2 unit
Number of families per household
Mean household size per Group 2 unit
Mean family size (total group size divided by number of married men)
Number of families per Group 2 unit
Mean household size per Group 1 unit
Political scale / complexity
Political and economic position of the group relative to other groups in the region (DW)
Type of social class distinction
Perogatives of leadership
Presence or absence of use of money within the society (DW)
Whether there is economic role specialization beyond simple age and gender differences
Community organization
"The functions and properties of structures with specific community-wide functions. These are not residences, nor are they multifunctional residences" (e.g., sweat lodge, dance house ...) (DW)
Ownership of resource locations
References from which datapoint is sourced. Refer to PDF file which contains complete bibliography.
References from which datapoint is sourced. Refer to PDF file which contains complete bibliography.
References from which datapoint is sourced. Refer to PDF file which contains complete bibliography.
References from which datapoint is sourced. Refer to PDF file which contains complete bibliography.
Not yet part of database
Loading Binford Variable Codes
Traceback (most recent call last):
File "./dplace_app/load.py", line 88, in
run(sys.argv[1], sys.argv[2])
File "./dplace_app/load.py", line 65, in run
load_bf_codes(csvfile)
File "/var/django/dplace/dplace_app/load/society_binford.py", line 181, in load_bf_codes
variable = VariableDescription.objects.get(label=label)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 310, in get
self.model._meta.object_name)
dplace_app.models.DoesNotExist: VariableDescription matching query does not exist.
Loading Binford Variable Values
Traceback (most recent call last):
File "./dplace_app/load.py", line 88, in
run(sys.argv[1], sys.argv[2])
File "./dplace_app/load.py", line 61, in run
load_bf_val(dict_row)
File "/var/django/dplace/dplace_app/load/society_binford.py", line 205, in load_bf_val
society = Society.objects.get(ext_id=ext_id)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 304, in get
num = len(clone)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in len
self._fetch_all()
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column dplace_app_society.xd_id does not exist
LINE 1: ...app_society"."id", "dplace_app_society"."ext_id", "dplace_ap...
^
Loading Geographic regions from shapefile
Loading Language tree ./datasets/trees/Afro-Asiatic.glotto.trees
Traceback (most recent call last):
File "./dplace_app/load.py", line 88, in
run(sys.argv[1], sys.argv[2])
File "./dplace_app/load.py", line 27, in run
load_glotto_tree(file_name)
File "/var/django/dplace/dplace_app/load/tree.py", line 60, in load_tree
language = get_language(taxon_name)
File "/var/django/dplace/dplace_app/load/tree.py", line 14, in get_language
language = Language.objects.get(iso_code__iso_code=taxon_name)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 304, in get
num = len(clone)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in len
self._fetch_all()
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column dplace_app_language.glotto_code_id does not exist
LINE 1: ...age"."name", "dplace_app_language"."iso_code_id", "dplace_ap...
^
Loading Language tree ./datasets/trees/Algic.glotto.trees
Traceback (most recent call last):
File "./dplace_app/load.py", line 88, in
run(sys.argv[1], sys.argv[2])
File "./dplace_app/load.py", line 27, in run
load_glotto_tree(file_name)
File "/var/django/dplace/dplace_app/load/tree.py", line 60, in load_tree
language = get_language(taxon_name)
File "/var/django/dplace/dplace_app/load/tree.py", line 14, in get_language
language = Language.objects.get(iso_code__iso_code=taxon_name)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/manager.py", line 151, in get
return self.get_queryset().get(*args, **kwargs)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 304, in get
num = len(clone)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in len
self._fetch_all()
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in exit
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/dplace_user/env_dplace/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column dplace_app_language.glotto_code_id does not exist
LINE 1: ...age"."name", "dplace_app_language"."iso_code_id", "dplace_ap...
^
and so forth for the rest of the trees.
Cheers, Hans

Hans-Joerg Bibiko
Max Planck Institute for the Science of Human History
Department of Linguistic and Cultural Evolution / Room 111
Kahlaische Str. 10	phone:	+49 (0) 3641 686 871
D-07745 Jena		fax:	+49 (0) 3641 686 990
Germany			e-mail:	bibiko [at] shh.mpg.de

Whoops! @Bibiko Forgot to tell you to run the command
python manage.py migrate dplace_app
To create the new tables/fields for the models. After that, run the load_all_datasets script. It will give you some errors (e.g., the lat long errors above), just ignore these and let the script run. It will take a while as there is a lot of data to read! Once it's done, everything should be working fine and I can check the site to make sure the data have been imported.
The glottolog tree display might look a bit funny though, as I haven't finished that.


The extension is asking me to manually save a file every time it needs to create one. It had been working fine until yesterday.

Type: Bug
Just enter every prompt that need to create a new file in the code base
Extension version: 0.24.2024121201
VS Code version: Code 1.96.2 (Universal) (fabdb6a30b49f79a7aba0f2ad9df9b399473380f, 2024-12-19T10:22:47.216Z)
OS version: Darwin arm64 24.1.0
Modes:
Prompt Details

Info: Using the Electron fetcher.
Info: Initializing Git extension service.
Info: Logged in as samuel-mr
Info: Successfully activated the vscode.git extension.
Info: Enablement state of the vscode.git extension: true.
Info: Successfully registered Git commit message provider.
Debug: [context keys] Updating context keys.

System Info

Item
Value

CPUs
Apple M2 (8 x 2400)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledwebgl: enabledwebgl2: enabledwebgpu: enabledwebnn: disabled_off

Load (avg)
2, 2, 2

Memory (System)
8.00GB (0.07GB free)

Process Argv
. --crash-reporter-id 2ecd5e8d-33af-4512-8b86-158be8e94718

Screen Reader
no

VM
0%

A/B Experiments
vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
vscaat:30438848
c4g48928:30535728
azure-dev_surveyone:30548225
vscrpc:30673769
2i9eh265:30646982
962ge761:30959799
pythonnoceb:30805159
pythonmypyd1:30879173
2e7ec940:31000449
pythontbext0:30879054
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
dvdeprecation:31068756
dwnewjupytercf:31046870
2f103344:31071589
nativerepl1:31139838
pythonrstrctxt:31112756
nativeloc1:31192215
cf971741:31144450
iacca1:31171482
notype1:31157159
5fd0e150:31155592
dwcopilot:31170013
stablechunks:31184530
6074i472:31201624

This is the window that shows me for each file, I have already restarted the app, I have reinstalled the app, I still have the problem

@samuel-mr  Yes, I occasionally encounter this issue myself. Often, pressing Discard and then re-editing will resolve it. I think this is an issue that must be fixed.


Update README.md

add --branch rel to git clone https://github.com/NVIDIA/TensorRT-LLM.git to clone the rel branch instead of main

Hi @MustaphaU Thanks for your contribution to TRT-LLM.
We still recommend the users try our latest improments/features on main branch instead of rel branch at this moment.
Sorry for the late response.


Disable animations when elements are added/removed but keep animations for when elements are reordered

I would like to disable animations when an element is added or removed from an array being rendered, but keep animations when the array is shuffled/reordered.
My understanding at the present is that we can enable/disable all animations as a whole, but not granularly. Is this possible?

Closing this one in favor of #153.


Column description

A lot of databases have non verbal or non descriptive column names that will for sure confuse semantically the LLM.
Would it be possible to add a method to describe or alias the column of a table ?
For example let's say I have a table called bsd with a column called cst: I want to alias them or describe them as building system and customer transaction.
Hopefully makes sense.

Hello, @priamai! :]
For our applications, we have chosen to define what these column names mean, when appropriate, as documentations, and train vanna to know what these column names are and what they mean. I would think such an approach would work for you as well.
Just remember to store each column name document chunk in a format that makes sense and preserves the hierarchical nature of your database, for instance like so: {schema, table, columnName, description}.
It is also likely that you might want to store chunks on table-level instead of row-level. Depends on your database and use case, I would think, what makes the most sense.
Let me know if you have further questions :]

Wait so for example:

I will do something like this?
vn.train(documentation="the table=Identity has column=id, the object unique identifier")
vn.train(documentation="the table=Identity has column=created, the date when the object was created")
....

@priamai did you make any progress in this issue? I'm also interested in it
Thanks!


Configuring Parallel Execution

Hi,
I'm trying to get my WebDriver specs2 tests to run with Parallel Execution.
How can I specify the number of worker threads that are used? It seems to be using about 4 (is this based on number of core's?) and unfortunately my machine melts with 4 browsers running.
Also, is it possible to have finer grained control about what is executed in parallel?
Specifcally I would like to be able to choose between running

all examples for a given spec in parallel (but each spec serially)
all specs in parallel (but each example serially)
everything in parallel

Many thanks,
Paul

More flexible control was implemented as part of:
https://github.com/harrah/xsbt/commit/eb2ec8c7815b225e556fc88d0c775b5ad01d77c3
and documented here:
https://github.com/harrah/xsbt/wiki/Parallel-Execution
Configuring parallel execution of examples is handled by the test framework and is not integrated with sbt's parallel execution (unfortunately.  this has been discussed on the mailing list in detail.).


PEX commands not working at all

Hey,
I am trying to follow the tutorial using the commands but all commands except pex debug result in a too many arguments error. I can't even send a group creation command.
Please help.
--
NN

Welp after trying to create group admin step by step over and over again and copy and pasting it 3 times over with the same error. I try the illogical and copy and paste it exactly one more time.
and it works.
still have no clue why it just decided to work after trying the same thing again and again, giving up, working on something else, then running it one more time.
I will close this you can reopen it if it seems i'm missing something

Without telling us which commands— exactly —that you were using, we couldn't help.  ;)

pex group failed with that error
pex user failed also
i tried all the commands in the tutorial and none worked except pex debug until i gave up and then tried copy and pasting exactly what was there for creating the admin group, which i had typed out manually 3 times before and it worked

What did you copy and paste?  Show me that text.

/pex group admin def true
then i added myself to the group with no errors copy and pasting /pex user  parent add group admin
and replacing name with my minecraft name

PEX 2.x?

yes im on sponge

OK.  @zml2008 will have to take a look at this stuff.  I handle 1.x issues.


can i use this for single table design?

can i use this for single table design?

AFAIK there's nothing that will stop you from doing it. I assume you could define more than one entity with the same table name. Is there something in particular you're concerned about?

Feel free to open a new issue if you encounter a concrete obstacle preventing from using Nova ODM with single-table design.


Add index for cloudevent.resource

Description
Adding an index on the cloudevent->>'resource' "column" in the events table.
Related Issue(s)

#316

Verification

[x] Your code builds clean without any errors or warnings
[x] Manual testing done (required)
[x] Relevant automated test added (if you find this hard, leave it and we'll help out)
[x] All tests run green

Documentation

[x] User documentation is updated with a separate linked PR in altinn-studio-docs. (if applicable)

I don't think this index will ever be used, because it's not very selective. Is there a change in the pipeline to ensure that the source field is used in most use cases? I suggest that we evaluate the index usage when such a change is in production.

Ref. slack call. I suggest you change to:
CREATE INDEX IF NOT EXISTS idx_events_cloudevent_resource_sequenceno
ON events.events ((cloudevent ->> 'resource'), sequenceno);
This index was created in 1 second in yt01, and it was used (proved by auto_explain) for a typical query with resource, after ("low" value to trigger worst case) and size.


[RFR] Fix default list state

It fixes a weird behavior:

Go on page 2 of posts list,
Refresh your browser,
Click on "Posts" menu item,
List is not refreshed.

Supersedes #264

Will be tested in #57.


Is it possible to use this toolkit to train multiple LORAs simultaneously?

This is a great toolkit. I have read the content available at the following URL:
https://github.com/center-for-humans-and-machines/transformer-heads/blob/main/notebooks/gpt2/joint_multitask_learning.ipynb.
However, it seems that there are multiple heads with a single LORA. I would like to confirm if it is possible to use this toolkit to train multiple LORAs simultaneously?
@rodrigobdz @yannikkellerde

I do not really see what the point of this would be. The idea here is to fine-tune one model to be able to do multiple tasks. If you want one LoRA per task, you should train a LoRA individually for each task (No need to add multiple heads to a Transformer)
Also, I am the sole maintainer of this project right now, so please do not ping anyone else.


Extension to other languages

Hello,
I have started to test your models and so far they seem to do a pretty good job. Thank you!
Any plans to build models for languages other than English? German and French would be great.
Best regards,
Ed

Yes, we are currently looking into that. We also want to create some models that do spelling correction at the same time.
Cannot give you an estimate of how long until we are ready to release them though.

I'm interested in using whitespace-correction for Vietnamese. How should I prepare training data?

In general you only need a large corpus of correctly spelled text in the language your interested in. For Vietnamese you could for example start with a dump of the Vietnamese Wikipedia.
The preprocessing pipeline then automatically injects whitespace errors during training.


Remove hostDNS from Talos config

Hello,
I am using talhelper for a while now, but recently discovered a bug/incompatibility issue with Talos v1.7.6. It seems like "hostDNS" is not supported anymore, but it is being added automatically by talhelper even when not in the talconfig.yaml.
It throws the following error:
error applying new configuration: 1 error occurred:         * <IP_ADDRESS>: rpc error: code = InvalidArgument desc = unknown keys found during decoding: machine:     features:         hostDNS:             enabled: true
Workaround is to remove the respective entry from the generated config files.

hostDNS is new feature of Talos v1.7. And Talhelper doesn't add that but it's added by default by Talos API when version is above 1.7. It's the same behavior when you do talosctl gen config.
I think the problem is you're using very old version of talosctl. Try updating your talosctl to latest version.

This is the relavant lines from upstream that does this: https://github.com/siderolabs/talos/blob/bc8bf9e8a865473011dcf5ad2df08c8ff1af5110/pkg/machinery/config/generate/init.go#L96-L101

I'm going to close this issue because of inactivity. Feel free to re open if you are not satisfied with my answer. Thank you.


Border line, seprator are not coming properly on Chrome browser

On Chrome browser, pdf is missing lines, dividers, separators etc...any idea what needs to be done

There is not enough information here for us to look into the issue. Please read https://github.com/mozilla/pdf.js/blob/master/CONTRIBUTING.md carefully and provide the details requested there, with a link to the PDF file being a minimal requirement.

its sensitive data else I wld have given...anyways , I will figure it out by own.


Remove the RequestId option within the protocol layer to check if anyone is using or setting it.

It seems like these are unnecessary implementation detail.

API change check
API changes are not detected in this pull request.

FYI @Jinming-Hu, @microzchang it doesn't look like this options field is used anywhere. Is it unnecessary and should it be removed?

FYI @Jinming-Hu, @microzchang it doesn't look like this options field is used anywhere. Is it unnecessary and should it be removed?
The RequestIdPolicy sets the x-ms-client-request-id header, if it isn't set by the client.

QQ: Does removing this field constitute a breaking change? It certainly appears that it does nothing, and thus it cannot be successfully used, but does removing something that has no effect count as a breaking change?
Note: I'm not saying that we bump the major version # for the removal - IMHO this counts in the category of "yeah, it's breaking but in practice, there's nobody actually using it, and anyway it doesn't actually work" .
But an acknowledgement in the changelog is probably in order.

@LarryOsterman the options being removed is within the protocol layer within rest_client.hpp, under _detatil namespace, and not exposed publicly. There is no public surface area breaking change (hence why I didn't put a CL entry). It's purely implementation detail clean-up.
That's why the API change check didn't notice any breaking change here.
The reason why the PR is in draft and not merged, is because we don't typically modify the generated code. I'll leave it up to storage folks if they'd like to modify the swagger/transform to make this clean-up change.
https://github.com/Azure/azure-sdk-for-cpp/blob/0bf6f88011f79809b0c505435c5660efcd2fb826/sdk/storage/azure-storage-files-datalake/inc/azure/storage/files/datalake/rest_client.hpp#L5-L6

Was using this as a proof-of-concept, to highlight unused implementation detail that can be safely removed.
FYI @Jinming-Hu, @microzchang


Failed to delete disk due to SoftLayer REST API changes

Due to some changes of SL REST API, softlayer-go could not delete disk as before.
https://sldn.softlayer.com/node/274103

Fixed by https://github.com/maximilien/softlayer-go/commit/785bd1461ffa4d496c31450b1912f434feb276d9


cannot install dependencies = [ "flask>=2.3.2", ] in toml file

This is happening in google colab environment. Causes:
ERROR: Cannot uninstall 'blinker'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

This is my toml file: https://github.com/anarchy-ai/LLM-VM/blob/main/pyproject.toml

That error is clearly something related to your environment and not related to Flask...


No setup instructions for Android

This is more of a complaint hahaha... But since there are instructions for iOS in the README it would be better to add Android instructions too, to keep it complete.
For a moment I thought Android was not supported by this library, so only showing instructions for iOS can be confusing.

@Positive07 I agree. Would you like to send a PR with an updated README?

Sorry but I haven't tried compiling this module with Android just yet. Instruction would probably be similar to the ones shown in section 2 of Android Guide for firebase-react-native but I should try it first.
If anyone has feedback please at least post it here, it will be really helpful


GET request to /programs endpoint returns object instead of array

For webersprinkler to be extensible, an array of program objects should be returned from a GET request to the /programs endpoint. Each program object should contain enough information to stand on its own. The reason an object is returned is related to the internal workings of the back end. I won't bother looking into the details now, but SOMETHING will need to be reworked. This is what I get for diving into the programming before defining a complete REST spec...

Perhaps the identifying letter should simply be embedded in the program object. Or, more in line with professional REST API's, there should be a random identification string associated with each program.


Plugin managers sometimes fail to update Conjure due to shallow clones

Errors:
fatal: Not possible to fast-forward, aborting.
a8686aa
NVIM v0.8.0-dev-737-g0b8bade49
(Was working till recently)

Ugh, not again. This happened last time I released. No force push, all I did was git merge --ff develop and git push. Why are plugin managers struggling with that?
And why has this started happening with recent releases despite my workflow not changing for years and years...

Original issue for this that I thought was over #404. So for some reason if I merge develop -> master and push now it breaks packer for people seemingly at random. I want to throw this over the fence and say it's an issue with Packer (which it really does feel like) but why is this not happening with every plugin? I don't think I'm doing anything special or wrong and haven't changed my behaviour in years (if ever since I started using git).
My theory is leaning towards Packer's optimisations clashing with specific git versions or repos, but that's a complete guess. I know it does shallow clones of some kind, maybe that's causing issues and can be turned off.

I fixed this issue for me by unshallowing the repository:
cd ...\nvim-data\site\pack\packer\start\conjure
git fetch --unshallow

And then I could update conjure successfully using PackerSync. If I understand the git documentation correctly (and supported by wbthomason/packer.nvim#33) the repository only needs to be unshallowed once.

I just ran into this again trying to update from 488014 to f50d4d.
When I manually run git pull, I get fatal: refusing to merge unrelated histories.
My local history looks like this:
git log --show-linear-break --decorate --oneline --parents master origin/master

f50d4d 8002df 6546ac (tag: v4.50.0, origin/master, origin/develop, origin/HEAD) Merge branch 'austinmlv-guile-output' into develop
6546ac 1f43cf Add more Guile testing tools
1f43cf c6d775 Fix / improve localleader defaulting in the school
                    ..........
8002df 8ba9bb Fix / improve localleader defaulting in the school
                    ..........
c6d775 8ba9bb guile: append stray output to log
8ba9bb 423761 Prevent leaky tap-fns by using a defonce #550
423761 595b0c Rework tap code so it supports babashka + only one mapping and no options
595b0c aa0ec4 Add queue for tap values in clojure client
aa0ec4 (grafted) More python sandbox shenanigans
                    ..........
488014 (grafted, HEAD -> master) Correct janet stdio module form name

Looking at this history, I'm not sure how merging is ever supposed to work for grafted commits in a shallow repository, nor do I understand why it only seems to happen in some repos (like this one) and not in others.

Thank you for doing all of this research, I'm sure this'll come in handy! I still don't think I'm doing anything non-standard and I've definitely seen some other repos / plugins complaining about this same issue. So I'm not sure if it's down to how I merge PRs into develop then into master or if it's down to things like my gpg signed commits.
It's also deeper in the git iceberg than I feel comfortable with. If we can work out exact reproduction steps maybe we can take that to the plugin managers and slowly change how they update things.
If there's some git pattern I'm doing that is normally fine but causes this issue I can change my workflows to try and avoid it too, it's just a little annoying.


PAYPAL

MOK

Hello,
Why do you want to add an empty PAYPAL file to this repository? I probably missed something?
:)

Hello, I close this PR as there is no response. Please reopen it if necessary ;)


Homebridge plugin

Hello. I used your app on Home Assistant, it worked great and even dumped the vacuum into HomeKit. Several months ago I switched to Homebridge and unfortunately there is no alternative there. The vacuum cleaner is very popular with tens of thousands of sales. Could you develop a full-fledged application for Homebridge or only mqtt commands for Homebridge so that later you could use for example the Homebridge Mqttthing plugin

I am not familiar with NodeJS (which is what homebridge based on) and I am not a user of homebridge itself. By the way, my package is enough documented to be ported on another programming language (especially if it has Amazon's API client). I don't have this port in plans.

I am not familiar with NodeJS (which is what homebridge based on) and I am not a user of homebridge itself. By the way, my package is enough documented to be ported on another programming language (especially if it has Amazon's API client). I don't have this port in plans.

Hey, i have ported the code to JavaScript and released a Plugin for Homebridge.
-> homebridge-tesvor
@opravdin thanks for your work!
greetings
Marcel

Hey, i have ported the code to JavaScript and released a Plugin for Homebridge.
-> homebridge-tesvor
@opravdin thanks for your work!
greetings
Marcel

Thank you very much for unsubscribing about this. This is very relevant to me. I have already installed your plugin


lncli getinfo "too many open files"

Background
I was having issues when running 'lncli getinfo' and getting 2 variations of the a message saying "too many files open" and was told in the slack to try raising my open file limit to 500000, previously the soft limit was 1024 and hard limit 4096.
Your environment

LND commit eobaa496
ubuntu 16.04
bitcoind 0.16
LND and bitcoind running under user bitcoin
0 open channels

bitcoin ~ ฿ ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 31321
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 500000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 8192
cpu time               (seconds, -t) unlimited
max user processes              (-u) 31321
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

bitcoin ~ ฿ ulimit -n
500000

bitcoin ~ ฿ ulimit -u
31321

Steps to reproduce
Restart LND, wait at least several hours (usually overnight) and run 'lncli getinfo'. When the issue first starts happening I will get the 'too many open files' message about 50% of the time, the longer I let my node run the more likely I get the message (~90%+)
Expected behaviour
Get a readout of the current lnd node info.
Actual behaviour
bitcoin ~ ฿ lncli getinfo
[lncli] rpc error: code = Unknown desc = unable to sync PoV of the wallet with current best block in the main chain: Post http://localhost:8332: dial tcp: lookup localhost: too many open files

bitcoin ~ ฿ lncli getinfo
[lncli] rpc error: code = Unknown desc = unable to get best block info: Post http://localhost:8332: dial tcp: lookup localhost: too many open files

Is there lots of connections if you check?
netstat -an will tell you that.
lsof -p <pid> where  is the process id of lnd will show you what files lnd has open. Keep in mind that a network connection is a file.
you can find the process ID of lnd if you do ps -fu bitcoin

I'd check to see if someone is attempting to drain your set of file descriptors. Is the bitcoind node remote?

I manually updated bitcoind to 0.16.1, the Ubuntu PPA is still at 0.16, after reading about something similar regarding 'too many open files' on bitcoin's github.  It has been running about 24 hours and no errors.
@vegardengen after I updated I ran the commands you suggested and it didn't look like there were that many files open, maybe a couple hundred at most.
@roasbeef I don't know how to check or verify if someone is draining my file descriptors.
Seems fixed now.

The file descriptors and connections will reset as you restart the programs. You should keep a watch on this - and keep these commands handy.
netstat -an and netstat -an | wc  -l - see if they are increasing.
lsof -p <pid of LND>

Still no errors, *fingers crossed.
netstat -an | wc -l is showing between 700-900.

The error is back again
netstat -an | wc -l showing 805
I don't see anything in the LND or bitcoind logs.

This is all I see in the dubug.log related to the RPC server.
2018-07-29 04:03:08 Received a POST request for / from <IP_ADDRESS>:59666
2018-07-29 04:03:08 ThreadRPCServer method=getblockhash
2018-07-29 04:03:08 Received a POST request for / from <IP_ADDRESS>:59668
2018-07-29 04:03:08 ThreadRPCServer method=getblock

Closing for now as the issue seems to have been resolved after upgrading bitcoind.


Allows configuring Faraday proxy

This PR allows one to configure a proxy through which the faraday requests will go to.
In my case, I'm using this to mock calls to the Paddle APIs from my test suite.

Thx for the PR!
It looks good to me. But it's a little difficult for me right now to test it, merge it, and then create a new version because I'm hiking the PCT right now and don't have access to a computer. Since it's not critical, I'll take care of it when I get back (around the end of September). I hope this works for you.

Thank you @nm for your comment, and it's completely fine to wait end of September.


Adds newline and wrap buttons to code view

Addresses #329
Allows long lines code view to be wrapped in the window and/or new lines to be rendered to aid in reading output
Before
After

https://stackstorm.slack.com/archives/community/p1492122082018664

Looks useful indeed :+1:

Hi @leleSahil, Is it feasible for you to include in this PR an enhancement to make the horizontal scroll bar much more prominent?
Note that in the following example where Chrome is being used, the scroll bar is very hard to pick out.  It's just to the left of the middle in this example.

Hi @enykeev, I can't seem to find any good icons on fontello. Is it absolutely necessary to use an icon from there? Or to use an icon at all?

No, not really necessary, icons just help something to look actionable without using default button appearance. Just style them to look like other buttons in st2web, that should be enough.

Hi @enykeev, I've changed the styling to match the other buttons in st2web and removed the tabs. Do you mind reviewing? We were hoping to get this included in your next release!

Thanks! I'll give it a look first thing tomorrow morning.

Here is few additions from me. @leleSahil please take a look and tell me if it works for you.

These look good to me! I'm sorry for the late response, I didn't see that you had responded with additional changes. Is it still possible to get this into the most recent release?

Yeah, this will be the part of 2.3.


help: force secrets upserting eventhough POLLER_INTERVAL_MILLISECONDS is set to higher interval

Hi guys, we have noticed that if we set POLLER_INTERVAL_MILLISECONDS=86400000 (24 hours) external secrets are not pulled from aws secrets manager and upserted to secrets immediately after deployment.
Use Case: Our secrets are db connection string and passwords which are not expected to change very often so we wanted to have poller interval once a day. we havent found way to force externalsecrets deployment to poll and upsert secrets immediately helm upgrade. it waits for 24 hours for even first time polling. please let us know if you need more details. thanks.

HI @avistech - this makes sense. It's probably pretty easy to fix, let us know if you'd like to submit a PR to fix it :-)

@jeffpearce thanks for quick response. May i know if it is good idea to fetch the secrets immediately for the first time and wait for POLLER_INTERVAL for the subsequent pulls (or) control the behaviour with a configuration parameter like env.POLL_SECRETS_ON_START=TRUE. i would appreciate your inputs. thanks.

I prefer fetching it the first time - adding another env variable just complicates the code, and I can’t think of a good reason not to always fetch it

@jeffpearce thanks again for prompt response. I just went through the code and noticed that adding forcePoll param to 'true' for ADDED case in daemon.js would suffice the requirement. Please let me know if this is good enough. thanks
    case 'ADDED': {
      this._addPoller(descriptor, true)
      break
    }

Yes, that seems like the right fix

Wasn't sure whether this was the best way to go or not (in #107), as this will upsert all secrets whenever the watch stream is disconnected or pod restarted or so, which might be fine. But then code can be simplified again as now the forcePull flag is unnecessary as its always set to true in all addPoller calls.
After #107 was merged it would however create the secrets right away if it doesn't find them as well as upsert as soon as the external secrets are modified. Might not really cover this use case, but I don't really see why one would need a forced upsert when kubernetes-external-secrets is modified?

@avistech, thoughts?

Hi, we are not aware of #107 changes but seems to be good for our use case. As we know, AWS pricing model is based on number of API calls per secret stored in secrets manager so we wanted the secrets to be pulled only once in a day or so. we have multiple microservices and if every service starts polling every 10 seconds its going to add costs to project because our secrets don't change so often. As  @Flydiverny mentioned, we are good if secrets are pulled right away if controller doesn't find them and wait for POLLER_INTERVAL for next poll. thanks.

I think we need to find some kind of improvement to implement as well though.
As say we have the 24h poll intervall, and the watch stream is disconnected within 24 hours everytime, we would never cause a new poll as the intervall would be reset everytime the watch stream is disconnected.

Made a draft for a possible solution in #133 :)

Thanks. we will wait for next release to upgrade our helm chart. :)


[TA] Bring back AAD nightly build instead of using private endpoint

Service team has AAD changes deployed to all regions.
It is time to bring back AAD nightly build

/azp run java - textanalytics - tests


Exporting menu not visible on IE

Hi,
I developed a reporting web app using your UI Grid. I have some troubles including export features: on Chrome and FF it works perfectly both CSV and PDF. On IE there's no export voice in the grid menu.
Is it a bug or I miss something?
Thanks a lot

I don't have IE to check, but does the tutorial work fine for you on IE?

Do you mean the example in the tutorial page? Yep, it seems work fine. So I think I missed something (config parameter, etc.) but I double check the code and I cannot find where the error is.

Yeah, but I can't see your code, so can't tell you what you've done wrong either.  :-)  Any chance of a plunker?

It's a little bit hard to create a jsfiddle/plunker because the controller which handle the grid creation/data handling will use some other services/external configuration params.
I already tried to create a "clean project" and using the grid in the same way I done in the controller, but no error in that case. So I think there's something strange which cause this issue.
May be a incompatibility with externale lib?

If it doesn't do it when you create a plunker, it would suggest that something in your code is causing it.

Information sought, no response for some months.  Closing.


Invalid character in path running dotnet ef

After installing Visual Studio 2017 RC running the command 'dotnet ef' results in an error of:
Illegal characters in path.
Parameter name: path
I don't know what path it is referring to or how to correct it.

Does specifying --verbose provide any additional context?

Well, that got me some more information but I'm still not clear as to what path it is referring to as having an illegal character.
Telemetry is: Enabled
projetfactory: MSBUILD_EXE_PATH = C:\Program Files\dotnet\sdk\1.0.0-preview4-004
233\MSBuild.dll
projetfactory: MSBuild project path = C:\Users\vbrown\Documents\Visual Studio 20
15\Projects\dotnetcore\ORCCore.SQLite\Core2ORCC.csproj
projecttoolscommandresolver: resolving commandspec from 0 Tool Libraries.
projecttoolscommandresolver: failed to resolve commandspec from library.
System.ArgumentException: Illegal characters in path.
Parameter name: path
at System.IO.PathInternal.CheckInvalidPathChars(String path)
at System.IO.Path.Combine(String path1, String path2)
at Microsoft.DotNet.Cli.Utils.EnvironmentProvider.<>c__DisplayClass7_0.b__2(String p, String s)
at System.Linq.Enumerable.d__784.MoveNext()    at System.Linq.Enumerable.FirstOrDefault[TSource](IEnumerable1 source, Func 2 predicate)    at Microsoft.DotNet.Cli.Utils.EnvironmentProvider.GetCommandPath(String comma ndName, String[] extensions)    at Microsoft.DotNet.Cli.Utils.PathCommandResolver.ResolveCommandPath(CommandR esolverArguments commandResolverArguments)    at Microsoft.DotNet.Cli.Utils.AbstractPathBasedCommandResolver.Resolve(Comman dResolverArguments commandResolverArguments)    at Microsoft.DotNet.Cli.Utils.CompositeCommandResolver.Resolve(CommandResolve rArguments commandResolverArguments)    at Microsoft.DotNet.Cli.Utils.CommandResolver.TryResolveCommandSpec(ICommandR esolverPolicy commandResolverPolicy, String commandName, IEnumerable1 args, NuG
etFramework framework, String configuration, String outputPath, String applicati
onName)
at Microsoft.DotNet.Cli.Utils.Command.Create(ICommandResolverPolicy commandRe
solverPolicy, String commandName, IEnumerable1 args, NuGetFramework framework, String configuration, String outputPath, String applicationName)    at Microsoft.DotNet.Cli.Utils.Command.Create(String commandName, IEnumerable
1 args, NuGetFramework framework, String configuration, String outputPath, Strin
g applicationName)
at Microsoft.DotNet.Cli.Program.ProcessArgs(String[] args, ITelemetry telemet
ryClient)
at Microsoft.DotNet.Cli.Program.Main(String[] args)

From the stack trace, this looks like a bug for the dotnet/cli repo. Could you  submit a new issue there so they can follow up?

This issue was moved to dotnet/cli#5317


Use Discord Commands instead of if/else

The Discord.js docs deprecate huge if/else statements for bot commands:
https://discordjs.guide/creating-your-bot/adding-more-commands.html#the-problem-with-if-else-if
I agree that the commands are quite confusing, so we should use the recommend Command  Handling with individual command files:
https://discordjs.guide/command-handling/#individual-command-files

Hi, assign it to me. And I think you should edit issue title, in this case we are not going to use discord commands but discord command files

Hey @yerlantemir I've assigned it to you!

done @Fubinator


[WIP] Add 1.17 deploy feature to skuba

This adds:

1.17 in the available versions for skuba, pointing to the right containers
Test the 1.16->1.17 in CI
Update cilium manifests for its new version (noop)
[TODO] Update unittests for 1.17.

To clarify for the future, the reason why the bump of coredns and etcd, it's to match the upstream recommended versions.

I will work with @mssola on updating those images, when they are available, I will update this.

Updated. We'll discuss the pattern package tomorrow.

This has removed the cilium requirements from the equation, dealing with them in the other PR.


LayerSwipe for GroupLayers

Hey Guys,
I tried to use GroupLayers (Basemap + overlying rasters) but it did not work. Would it be possible to add this option?  Or is it already possible but I am just to dumb to do it?
Thank you!

Layer groups do not have prerender and postrender events, The individual layers in a group need to be handled https://codesandbox.io/s/quizzical-water-bhuei?file=/main.js

Thanks for the code and explanation!


Merged dev and bumped webpack

That's fine, it seems like I made a mistake while merging, because I can't see those merged commits, so maybe it's for the best 😃.

@Dattaya Aah okay :smile:


Wait for apiman to be ready

Add a feature to wait for apiman to be ready, with a given timeout. Use rnorth/ducttape for retry.
This will help when scripting the deployment of configuration to a server that has just been started.

Done in https://github.com/apiman/apiman-cli/commit/4f1907b5c4e7400b2df1b5ddfba760c3019cd0a7


Deepview redirects to default / desktop location instead of app on new iPadOS Beta

Clicking a branch link doesn't take you to the iOS app if the device is running iPadOS, it just takes you to the fallback location (desktop in our case). It works fine on all iOS 13 versions.
Am I missing something or is iPadOS not supported yet?
Thanks

updating to new iOS beta seems to have fixed it

Hi, it looks like this is still happening on an iPad running iPadOS.
If the url scheme / universal link works, you're taken to the app, all good.
If you're taken to a deepview, you will be redirected to the desktop url.
Is iPadOS treated as a desktop platform and not standard iOS?
Thanks

Could you open a new ticket for this?  I'll open an internal ticket to check on this.  iPadOS should not be treated as a desktop, however it self reports as a desktop so there could be an issue server side determining what device type it is.
Also are you able to provide OS version, browser version and possibly a sample link?

Sorry for the delay.
OS is IpadOS 13.1.1
Browser is Safari, not sure how to get the version number on an iPad.
I have a link ready, is there a way I can send it to you privately?

@NumaNumaNuma
Thanks for the info.  You can send the link to<EMAIL_ADDRESS>I'm hoping to devote time to this early in the next sprint.

Great, I sent you an email with the link 👍

Hi @echo-branch,
Just checking in. Have you been able to reproduce / confirm the problem? Or is it something I'm doing wrong?
I'd love to know if I should keep digging or just wait for a fix :)
Thanks

I have the same problem on an iPad with iPadOS 13.4.1.
When I tap a branch link it redirects to the "text-me-the-app page", which is the default redirect for the Desktop.

@PreludeAndFugue
Just to update here.  There's an issue with us misidentifying iOS 13 iPads that have the "request desktop site" setting in Safari changed.  There's a ticket to fix this in the current sprint, though it's taking longer than expected.
You may be able to work around it by defaulting to the iPad version of the site on failure to identify, but that might not work if you have a desktop site too.

Hi @echo-branch, any news? 😁

@echo-branch Please update on your progress in this issue, Even I am facing the same issue.

@NumaNumaNuma why have you closed this issue? Is it resolved for you ?

I closed it in 2019 when I thought it worked, turns out it was still broken. I'll re-open!

@echo-branch please confirm when this issue will be picked up by your team.

@echo-branch any news on this? 😅

Hey sorry for not responding earlier, I missed the ping since you reopened the issue.  The original backend ticket is marked fixed, but I'll follow up with team.

I reopened a ticket for the backend team.

@echo-branch Friendly check-in. Any updates?

Hey. So we confirmed that the original fix wasn't completed. The server team has been completely devoted to iOS 14.5 work, so they haven't gotten it yet. I am hoping it makes it into the next sprint now that iOS 14.5 is out.

Any updates on this? Is it on the map?

hi guys apparently, this IPadOS issue was affecting the entire Branch SDK, since it was a change from Apple's move.
Now on our Branch Dashboard, you can set up the IpadOS deep linking there.
Please have a look at the attached image :)
https://flic.kr/p/2n7qFMS


Typed property Saade\FilamentFullCalendar\Widgets\FullCalendarWidget::$record (on drag and drop)

Hi, thanks for making this plugin!
I've followed the readme and I'm trying to update the dates on drag and drop.
I have a Calendar page which has the CalendarWidget.php
I have at the top public Model | string | null $model = JobTask::class;, although have tried with and without it. I am hoping to target many models, let me know if this is an issue.
Using use Saade\FilamentFullCalendar\Actions; I get the error Typed property Saade\FilamentFullCalendar\Widgets\FullCalendarWidget::$record, with no model updates.
Using use Filament\Actions; it will work, although have App\Filament\Dashboard\Widgets\CalendarWidget::App\Filament\Dashboard\Widgets\{closure}(): Argument #1 ($record) must be of type Illuminate\Database\Eloquent\Model, null given, called in /var/www/laravel/vendor/filament/support/src/Concerns/EvaluatesClosures.php on line 35 after the event has finished.

I am using this code, note for the arguments I had appended |null for the use Filament\Actions; to work. Without them there is an error thrown as they do not exist.
I figured it's best to use use Saade\FilamentFullCalendar\Actions; but I must be overlooking something or missed a step with the setup. This is the CalendarWidget.php file. Note I had tried to work around this by using the switch and find model statements, but no luck.
<?php

namespace App\Filament\Dashboard\Widgets;

use App\Filament\Dashboard\Resources\JobResource;
use App\Filament\Dashboard\Resources\JobTaskResource;
use App\Models\Job;
use Carbon\Carbon;
use Filament\Actions;
use Filament\Forms\Form;
use Illuminate\Database\Eloquent\Model;
use Saade\FilamentFullCalendar\Widgets\FullCalendarWidget;
use App\Models\JobTask;

class CalendarWidget extends FullCalendarWidget
{
    /**
     * FullCalendar will call this function whenever it needs new event data.
     * This is triggered when the user clicks prev/next or switches views on the calendar.
     */
//    public Model | string | null $model = JobTask::class;

    public function fetchEvents(array $info): array
    {
        // You can use $info to filter events by date.
        // This method should return an array of event-like objects. See: https://github.com/saade/filament-fullcalendar/blob/3.x/#returning-events
        // You can also return an array of EventData objects. See: https://github.com/saade/filament-fullcalendar/blob/3.x/#the-eventdata-class

        $jobTasks = JobTask::query()
            ->where('start_date', '>=', $info['start'])
            ->where('end_date', '<=', $info['end'])
            ->get()
            ->map(
                fn (JobTask $event) => [
                    'id' => $event->id,
                    'calendar_type' => 'job-task',
                    'title' => $event->name,
                    'start' => $event->start_date,
//                    'end' => $event->end_date,
                    'backgroundColor' => 'blue',
                    'eventBorderColor' => 'blue',
                    'displayEventTime' => false,
                    'url' => JobTaskResource::getUrl(name: 'view', parameters: ['record' => $event]),
                ]
            )
            ->all();

        $jobs = Job::query()
            ->where('start_date', '>=', $info['start'])
            ->where('deadline_date', '<=', $info['end'])
            ->get()
            ->map(
                fn (Job $event) => [
                    'id' => $event->id,
                    'calendar_type' => 'job',
                    'title' => $event->name,
                    'start' => $event->start_date,
//                    'end' => $event->deadline_date,
                    'backgroundColor' => 'darkgrey',
                    'eventBorderColor' => 'darkgrey',
                    'displayEventTime' => false,
                    'url' => JobResource::getUrl(name: 'view', parameters: ['record' => $event]),
                ]
            )
            ->all();

        return array_merge($jobTasks, $jobs);
    }

    protected function modalActions(): array
    {
        return [
            Actions\EditAction::make()
                ->mountUsing(
                    function (Model|null $record, Form|null $form, array $arguments) {
                        dd($record,$form,$arguments);
                        switch ($arguments['event']['extendedProps']['calendar_type']) {
                            case 'job':
                                $record = Job::find($arguments['event']['id']);
                                $record ->update([
                                    'start_date' => Carbon::parse($arguments['event']['start']),
                                ]);
                                break;

                            case 'job-task':
                                $record = JobTask::find($arguments['event']['id']);
                                $record->update([
                                    'start_date' => Carbon::parse($arguments['event']['start']),
                                ]);
                                break;
                        }
                    }
                ),
        ];
    }

}

Actually the solution for this was rather simple and I had had overridden the drag and drop event here - https://github.com/saade/filament-fullcalendar/blob/3.x/src/Widgets/Concerns/InteractsWithEvents.php
Originally the documentation assumes there is just one Model type Event. Having more than one Model type and having some custom code to parse the events and save them works much better.
This is what I had ended up using.
    public function onEventDrop(array $event, array $oldEvent, array $relatedEvents, array $delta): bool
    {
        $model = null;

        switch ($event['extendedProps']['calendar_type']) {
            case 'job':
                $model = Job::findOrFail($event['id']);
                $model->deadline_date = Carbon::parse($event['end']);
            break;

            case 'job-task':
                $model = JobTask::findOrFail($event['id']);
                if ($event['end']) {
                    $model->end_date = Carbon::parse($event['end']);
                }
            break;
        }

        $model->start_date = Carbon::parse($event['start']);
        $model->save();

        return false;
    }


Add Button Field to Generic Widget

The button field is just what it sounds like: It lets you add a generic button to any widget, with the action being a method of the widget (or any other function, really.)
We're using this to be able to tare flows in our MFCs.

Ah oops, I branched this off the automation logging branch and shouldn't have. Will close and re-PR.


Can not start magic mirror with calendar module any longer

Dear all,
after a while I tried to start my magic mirror again with the calendar module but out of no reason I currently get the following error:
[04.08.2021 12:13.12.814] [LOG]   Starting MagicMirror: v2.16.0
[04.08.2021 12:13.12.821] [LOG]   Loading config ...
[04.08.2021 12:13.12.826] [LOG]   Loading module helpers ...
[04.08.2021 12:13.12.827] [LOG]   No helper found for module: alert.
[04.08.2021 12:13.12.948] [LOG]   Initializing new module helper ...
[04.08.2021 12:13.12.949] [LOG]   Module helper loaded: updatenotification
[04.08.2021 12:13.12.950] [LOG]   No helper found for module: clock.
[04.08.2021 12:13.13.364] [ERROR] WARNING! Could not load config file. Starting with default configuration. Error found: Error: Cannot find module 'crypto-js'
[04.08.2021 12:13.13.365] [LOG]   Loading module helpers ...
[04.08.2021 12:13.13.366] [LOG]   No helper found for module: alert.
[04.08.2021 12:13.13.367] [LOG]   Initializing new module helper ...
[04.08.2021 12:13.13.375] [LOG]   Module helper loaded: updatenotification
[04.08.2021 12:13.13.376] [LOG]   No helper found for module: clock.
[04.08.2021 12:13.13.380] [ERROR] Whoops! There was an uncaught exception...
[04.08.2021 12:13.13.385] [ERROR] { Error: Cannot find module 'crypto-js'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:636:15)
    at Function.Module._resolveFilename (/home/pi/MagicMirror/node_modules/module-alias/index.js:49:29)
    at Function.Module._load (internal/modules/cjs/loader.js:562:25)
    at Module.require (internal/modules/cjs/loader.js:692:17)
    at require (internal/modules/cjs/helpers.js:25:18)
    at Object.<anonymous> (/home/pi/MagicMirror/node_modules/digest-fetch/digest-fetch-src.js:10:18)
    at Module._compile (internal/modules/cjs/loader.js:778:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)
    at Module.load (internal/modules/cjs/loader.js:653:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:593:12) code: 'MODULE_NOT_FOUND' }
[04.08.2021 12:13.13.386] [ERROR] MagicMirror will not quit, but it might be a good idea to check why this happened. Maybe no internet connection?
[04.08.2021 12:13.13.387] [ERROR] If you think this really is an issue, please open an issue on GitHub: https://github.com/MichMich/MagicMirror/issues

After removing the calendar module, everything works as expected.
Even with the default config.js the error appears.
Any idea how to solve this ?
Even the npm run config:check is ok.
Thanks in advance !

missing crypto-js lib
[04.08.2021 12:13.13.364] [ERROR] WARNING! Could not load config file. Starting with default configuration. Error found: Error: Cannot find module 'crypto-js'
do
cd ~/MagicMirror
npm install crypto-js

Okay.
Without the calendar module everything works as expected.
So this made me believe that the calendar module needs it somehow.
But the hint with the dependency of a dependency might be the reason...

Are you using the default calendar module?

Yes.
I even tried it with the default configuration file provided - but even with this configuration file the error arises.
So the library is somewhere needed and I'm gonna install it as @sdetweil suggested.

@jupadin @MichMich
its needed by
node_modules/digest-fetch/digest-fetch-src.js

If I run npm ls --all it doesn't show crypto-js, so it doesn't seem to be a (deep) dependency of MagicMirror.

@sdetweil, that seems to be crypt not crypto-js, is it?

@MichMich
the error message is
 Error found: Error: Cannot find module 'crypto-js'

maybe the server side changed their encryption type..

Yes, I know. But I can't seem to find which module is trying to use it. Since it isn't in the dependency tree.

Maybe md5 used crypto.js and switched to crypt?

@jupadin what you could try is reinstalling your node modules to make sure all your dependencies are up to date:
# from within your ~/MagicMirror folder:

rm -Rf node_modules && npm install

calendarfetcher.js uses digest-fetch

Yes, which uses md5, which uses crypt. I can't seem to find any module that has crypto.js as a dependency.

Sorry. You're correct. But same story! crypt is the dependency. not crypto (notice the o at the end).

md5 mentions crypto-js in the README

Yes, but only that it was based on that implementation:

The original implementation can be found in the CryptoJS project.

It is not a dependency. So I wonder which module is trying to load the crypto-js module.

well, the stack trace says
 at Object.<anonymous> (/home/pi/MagicMirror/node_modules/digest-fetch/digest-fetch-src.js:10:18)

and turning off calendar stops the error..

Yeah. But I'm trying to understand why (especially since @jupadin is the only user with this issue).

I just updated the node version as well as the npm version with a clean
# from within your ~/MagicMirror folder:

rm -Rf node_modules && npm install

which fixes the error...
But anyway, might be interesting where this crypto-js library comes from

Found it! It was a dependency of an old digest-fetch:
https://github.com/devfans/digest-fetch/pull/24/files

So by updating digest-fetch the problem went away for you. And since I already had the new version. It didn't appear in my dependency tree.

Can I close this issue?

Yes. Thank you very much for your time and investigation !!! :)


Screen quarter size on iOS

Please ensure you have given all the following requested information in your report.
Issue details
The screen when running the iOS version is a quarter of the size it should be when first starting the app
Reproduction steps/code
I first noticed this in my app and tried to reproduce in a newly created app with success. This can be reproduced easily using the libGDX setup app. Just create a new project and launch in iOS. The problem occurs in both the simulator and on the device.
Change the line that draws the logo to
batch.draw(img, 0, 0, Gdx.graphics.getWidth(), Gdx.graphics.getHeight());
The logo will only fill half the screen. If you draw the logo with double dimentions then it's clipped to the bottom left quarter.

Version of LibGDX and/or relevant dependencies
1.9.13-SNAPSHOT and 1.9.12
Please select the affected platforms

[ ] Android
[x] iOS
[ ] HTML/GWT
[ ] Windows
[ ] Linux
[ ] MacOS

Hey @WillCalderwood, that is expected behaviour. The methods you should be using in this case are Gdx.graphics.getBackBufferWidth() & getBackBufferHeight(). Alternatively, use a viewport. Specifically for iOS, this was changed in 1.9.12, see the corresponding changelog.

@crykn  Thanks. I thought I must be missing something obvious but a quick Google didn't reveal anything. I'll look at updating the new project template to use a viewport so it behaves the same on both Android & iOS.


feat: support preloading also on legacy bundle

Description
Support preloading also on legacy bundle.
A test case is included for a utility function I'm using.
closes #9902 and even more (since it's also preload JS scripts)
fixes #2062
closes #5901
fixes #10285
Additional context
As mentioned in #9902 and #10285, by solving the issue with this PR, it makes the legacy build support for SvelteKit I'm working on(sveltejs/kit#6265) to load correctly the CSS files when navigating(and more correctly, on preloading) from one page to the other.

What is the purpose of this pull request?

[X] New Feature
[ ] Documentation update
[ ] Other

Before submitting the PR, please make sure you do the following

[X] Read the Contributing Guidelines.
[X] Read the Pull Request Guidelines and follow the Commit Convention.
[X] Check that there isn't already a PR that solves the problem the same way to avoid creating a duplicate.
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123).
[ ] Ideally, include relevant tests that fail without this PR but pass with it.

Help: Don't know why vue legacy tests are failing.

It seems this image is not being loaded when doing pnpm build && pnpm preview in playground/vue-legacy.
https://github.com/vitejs/vite/blob/4158b98c8fc30070967de1176f4b1c252d275386/playground/vue-legacy/module.vue#L9

It seems this image is not being loaded when doing pnpm build && pnpm preview in playground/vue-legacy.
https://github.com/vitejs/vite/blob/4158b98c8fc30070967de1176f4b1c252d275386/playground/vue-legacy/module.vue#L9

Tnx! I had realized that the root cause for the failure is because I'm fighting with Vite way to inline the CSS on legacy.
But the immediate question now is, why?
I have commented this in the original issue I had opened:
https://github.com/vitejs/vite/issues/9902#issuecomment-1236194069

I tested with Chrome 16 and it worked greatly. 👍 (when minify is disabled)

Test now, it's working on my machine also after minimization, including testing.
Note: As said, I had stripped out the XHR call. This results with the side effect that if the network resource(the CSS file) isn't available, then no exception will be raised on legacy.
If this is crucial I can return this extra call.
Note2: I had modified a little bit the directory structure of the testing, to include the code samples. Hope it's fine

Now I think it's ready.
Suggest to merge #9970 first, both for credits and test units.

I had removed the fix of #9967 from this PR, since it's handled better in #9970.
Now both #9970 and this PR are independent, and merging one won't affect the other.
Note: Need to still implement in the future the suggestion appears in the discussion of https://github.com/vitejs/vite/pull/9970#discussion_r969505865, but this can be done later in a different PR, as this suggestion is nice to have.
So now I'm ready for review&merge.

This implementation depends on regexs very heavily and I felt a bit fragile.
So I have a suggestion, how about using instantiate hook of SystemJS? In this way, I think we could avoid regexs.
In entry chunks:
if (!window.__viteLegacyDeps) {
  window.__viteLegacyDeps = {}

  const originalInstantiate = System.constructor.prototype.instantiate
  System.constructor.prototype.instantiate = function (args) {
    // preload using the information of `window.__viteLegacyDeps`
    return existingHook.call(this, args)
  }
}

In each chunks:
window.__viteLegacyDeps['chunk file'] = [/* dep list */]

This implementation depends on regexs very heavily and I felt a bit fragile.

This is why I added testing snapshots😃
I agree however about the principle. ESModules import are scanned well by Vite. Do you think we can somehow generate a map at the renderChunk stage(or such) between the es-formatted chunk to their deuended dynamically imported (other) chunks, and then use it directly for writing the dependency list?

In each chunks:
window.__viteLegacyDeps['chunk file'] = [/* dep list */]

How do we get the dependency list here? Feels like we need to have the regexps (or an alternative solution) for this again.

Do you think we can somehow generate a map at the transform/renderChunk stage(or such) between the es-formatted chunk to their (well-)detected dynamically imported (other) chunks, and then use it directly for writing the dependency list?

I think we can't. Dependency graph can only be obtained in renderChunk/generateBundle, but in those hooks the code is already transformed to SystemJS.

How do we get the dependency list here? Feels like we need to have the regexps (or an alternative solution) for this again.

I guess we can use dynamicImports property of ChunkInfo to get what is dynamic-imported. Tracing static-imports  could be done like we do now.
https://rollupjs.org/guide/en/#generatebundle

Do you think we can somehow generate a map at the transform/renderChunk stage(or such) between the es-formatted chunk to their (well-)detected dynamically imported (other) chunks, and then use it directly for writing the dependency list?

I think we can't. Dependency graph can only be obtained in renderChunk/generateBundle, but in those hooks the code is already transformed to SystemJS.

How do we get the dependency list here? Feels like we need to have the regexps (or an alternative solution) for this again.

I guess we can use dynamicImports property of ChunkInfo to get what is dynamic-imported. Tracing static-imports could be done like we do now. https://rollupjs.org/guide/en/#generatebundle

@sapphi-red, continuing the logic in my previous comment, do you think that any (breaking?) change of Vite 4.0 can help us here?

I deepen into Rollup chunk information, but it seems to tell us only about the dependencies sources, but not on the location on the chunk that the "import" is used, and parseImports/parseImportsSystemJS give us this extra information.

We only need that what will be dynamically imported in that chunk. I'm suggesting to declare the following code in the head of the chunk, not aside of the import.
window.__viteLegacyDeps['chunk file'] = [/* dep list */]

For example, transform this code
// main.000.js ------------------------------
System.register([], function (exports, module) {
  module.import('foo.111.js')
  module.import('bar.222.js')
});

// foo.111.js ------------------------------
System.register([], function (exports, module) {
  console.log('foo.111.js')
});

// foo.111.css ------------------------------
/* css content */

// bar.222.js ------------------------------
System.register([], function (exports, module) {
  console.log('bar.222.js')
  module.import('foobar.333.js')
});

// foobar.333.js ------------------------------
System.register([], function (exports, module) {
  console.log('fioobar.333.js')
});

to
// main.000.js ------------------------------
if (!window.__viteLegacyDeps) {
  window.__viteLegacyDeps = {}

  const originalInstantiate = System.constructor.prototype.instantiate
  System.constructor.prototype.instantiate = function (args) {
    // preload using the information of `window.__viteLegacyDeps`
    return existingHook.call(this, args)
  }
}

window.__viteLegacyDeps['foo.111.js'] = ['foo.111.css']
window.__viteLegacyDeps['bar.222.js'] = []

System.register([], function (exports, module) {
  module.import('foo.111.js')
  module.import('bar.222.js')
});

// foo.111.js ------------------------------
System.register([], function (exports, module) {
  console.log('foo.111.js')
});

// foo.111.css ------------------------------
/* css content */

// bar.222.js ------------------------------
window.__viteLegacyDeps['foobar.333.js'] = []

System.register([], function (exports, module) {
  console.log('bar.222.js')
  module.import('foobar.333.js')
});

// foobar.333.js ------------------------------
System.register([], function (exports, module) {
  console.log('fioobar.333.js')
});

How then can we overcome the removing process of pure css imports?

How about injecting window.__viteLegacyPureCss = [/* pure css deps */] and skipping them in instantiate when preloading?

continuing the logic in my previous comment, do you think that any (breaking?) change of Vite 4.0 can help us here?

I'm not able to track the changes deeply but I guess no.

Took some work but finally made it with SystemJS hooks!
Ready for review @sapphi-red :-)

Sorry that it took a long time.
I tried this PR out and found that it doesn't work when there's a dynamic import in a dynamic imported chunk.
For example, in https://github.com/vitejs/vite/pull/9920/commits/211f4a7ca9e1659683957a5ae82ee4bef2499dff, "async2" should be blue but it isn't in legacy.
I guess this is because this PR's implementation only injects a single __VITE_PRELOAD_LIST__. __VITE_PRELOAD_LIST__ has to exist in every chunk.

I'm sorry that it took a long time.
I tried this PR out and found that it doesn't work when there's a dynamic import in a dynamic imported chunk. For example, in 211f4a7, "async2" should be blue but it isn't in legacy. I guess this is because this PR's implementation only injects a single __VITE_PRELOAD_LIST__. __VITE_PRELOAD_LIST__ has to exist in every chunk.

What is the way to go with that?
I also wasn't sure about the global declaration policy in Vite, and the chunk loading.

What is the way to go with that?

I think injecting the following code to every chunk in generateBundle hook would work.
${preloadListMarker}.forEach((data) => {
  ${preloadLegacyImportsToken}[assetsURL(data[0], import.meta.url)] = {
    deps: data[1],
    pure: data.length === 3
  }
})

I also wasn't sure about the global declaration policy in Vite, and the chunk loading.

This PR already depends on window.__viteLegacyImports. What do you mean by "global declaration policy"?


Currency updating should immedietly update currency_exchange_last_refresh and currency_exchange_historical_last_refresh key

The currency updater module should immedietly update the currency_exchange_last_refresh and currency_exchange_historical_last_refresh key to ensure that multiple running tasks that implement the currency module don't start updating all nearly at the same time just because the value of the key is outdated.

Then we should also make sure we do some kind of "update this only if it still has the old value", otherwise you can still have two tasks updating the currency values at the same time if they both read and then write in the correct order.

yes, i was in for a lock as well. @j4k4 what do you think? To lock or not to lock, that is the question. :lock: :closed_lock_with_key: