Fix/Streamline Upload Form & Processing

[x] https://whgazetteer.org/datasets/create/

[x] Add modal Spinner on submit.
[x] Licence information ought to have a checkbox acknowledging acceptance.
[x] Allow repeat submission of label if upload fails - i.e. tidy up properly before returning to form.
[x] Map Wikidata, :aat, and other types to GeoNames fclasses, e.g. {'label': 'human settlement', 'identifier': 'wd:Q486972'}
[x] Delete uploaded Essay file
[x] Add optional default value inputs for fclasses, timespans; each should have a choice of augment, replace, or where missing.

[x] Add to form with validation checks
[x] JSON: datasets.insert.ds_insert_json
[x] Delimited: datasets.insert.ds_insert_delim

[x] Place detail fails to load on http://localhost:8001/datasets/nnn/places (minmax function fails if either start or end is missing from a timespan)
[x] Delimited insertion is not atomic (actually more a problem with failure sometimes to delete dataset on insertion failure)
[x] Need to add columns to df for fclasses/start/end if set as defaults
[ ] Intermittent duplicate id on insert error
[x] Refactor ds_insert_json:

[x] Combine objs, data_mappings, and bulk_create_operations
[x] dbcount is a redundant check
[x] src_id=feat['@id'] if uribase in ['', None] else feat['@id'].replace(uribase, ''), should remove only from start of string

[x] Cancel return URL is incorrect in some circumstances.

[ ] https://whgazetteer.org/datasets/1506/places Gives count incorrectly as "None".
[ ] https://whgazetteer.org/datasets/1506/delete also reports "None place records"
[ ] http://localhost:8001/datasets/177/status needs space after "Contributed by:"
[x] requirements.txt needs addition of odfpy==1.4.1 in order to handle upload of .ods data files.
[ ] Zipped sample datasets fail upload due to absence of "valid 'fclasses' or 'aat_types'"
[ ] If a delimited dataset includes any features with aat_types, validation requires all to have aat_types even if they have fclasses.

Re-think of Dataset Upload & Validation

[x] Remove options for default fclasses and timespans.
[x] For the simplest possible UX, consider removing all other optional (metadata) inputs (these values can be added later, before publication).

Resources
/datasets/templates/datasets/dataset_create.html
datasets.views.DatasetCreate
datasets.forms.DatasetUploadForm

Notes
requirements.txt needs addition of odfpy==1.4.1 in order to handle upload of .ods data files.

Upon reviewing the timespansReduce function and the example data you provided, I can identify potential issues that might cause the function to fail or produce unexpected results:

Handling of Optional Keys:

The function assumes that each timespan dictionary (ts) has both 'start' and 'end' keys. However, in your example data, some timespans do not have an explicit 'end' key, which could lead to a KeyError when attempting to access ts['end'].

Empty Values:

The function attempts to handle cases where the 'end' key might be present but empty (''). It assigns e to s in such cases. However, if ts['end'] is not present at all, this could lead to a KeyError or an unintended default behavior.

Error Handling:

The function lacks robust error handling mechanisms. If any of the expected keys ('start' or 'end') are missing or in unexpected formats in the input data (ts), it might raise exceptions or produce incorrect results.

Example Analysis:
Let's examine the example data you provided and how it might interact with the timespansReduce function:
{
  'type': 'Feature',
  '@id': 'IV:IV1680-001-01',
  'geometry': {
    'type': 'GeometryCollection',
    'geometries': [
      {'type': 'Point', 'certainty': 'certain', 'translation': {'distance': 1.2, 'tps_reference': [-2.4354, 51.004]}, 'coordinates': [-2.4182265, 51.0028024]},
      {'type': 'Point', 'citations': [{'year': '1680', '@id': 'doi:10.5281/zenodo.4749505'}], 'certainty': 'uncertain', 'coordinates': [-2.4, 51.06]}
    ]
  },
  'when': {
    'timespans': [
      {'start': {'latest': '1680'}, 'end': {'earliest': '1680'}}
    ]
  },
  'relations': [
    {'relationType': 'gvp:broaderPartitive', 'relationTo': 'h-counties:Somerset', 'label': 'part of Somerset (Historical UK County)', 'when': {'timespans': [{'end': {'earliest': '1974'}}]}},
    {'relationType': 'gvp:broaderPartitive', 'relationTo': 'h-hundreds:Horethorne_Somerset', 'label': 'part of Horethorne, Somerset (Historical UK Jurisdiction)', 'when': {'timespans': [{'end': {'earliest': '1974'}}]}}
  ],
  'types': [{'label': 'human settlement', 'identifier': 'wd:Q486972', 'sourceLabels': [{'label': 'settlement-gentlemen-1'}]}],
  'links': [
    {'type': 'closeMatch', 'identifier': 'GB1900:58cefe072c66dcf8fa046d2f'},
    {'type': 'closeMatch', 'identifier': 'osm:917415514'},
    {'type': 'closeMatch', 'identifier': 'wd:Q21406516'}
  ],
  'properties': {
    'glyphs': ['settlement-gentlemen-1'],
    'title': 'Abbas Comb',
    'ccodes': ['GB'],
    'id': 'IV:IV1680-001-01',
    'dataset': 'Index Villaris, 1680',
    'county': 'Somerset',
    'hundred': 'Horethorne (Somerset)'
  },
  'names': [
    {'toponym': 'Abbas Comb', 'citations': [{'year': '1680', '@id': 'doi:10.5281/zenodo.4749505'}], 'when': {'timespans': [{'start': {'latest': '1680'}, 'end': {'earliest': '1680'}}]}},
    {'toponym': 'Abbas Combe', 'citations': [{'@id': 'GB1900:58cefe072c66dcf8fa046d2f', 'year': '~1900'}, {'@id': 'wd:Q21406516', 'year': '2023'}], 'when': {'timespans': [{'start': {'latest': '1913'}, 'end': {'earliest': '2023'}}]}}
  ]
}

Potential Issues in Example Data:

The timespansReduce function expects ts to have 'start' and optionally 'end' keys. In your example:

The first timespan under 'when' has both 'start' and 'end' keys, which is handled correctly.
However, in the 'relations' list, each relation might have a 'when' key with 'timespans', but it's not guaranteed to have an 'end' key. This could lead to exceptions when trying to access ts['end'].

Recommendations:

Ensure Key Existence: Modify the timespansReduce function to check if keys like 'start' and 'end' exist in ts before accessing them. This prevents KeyError exceptions.

Default Values: Handle cases where 'end' is missing or empty ('') more gracefully. Consider defaulting end to None or s if necessary.

Iterate Safely: Use safe iteration techniques (like get() method or in keyword checks) to safely access nested dictionary keys without causing errors.

Here's a revised version of timespansReduce function with some enhancements based on the analysis:
def timespansReduce(tsl):
    result = []

    for ts in tsl:
        start = ts.get('start', {})
        end = ts.get('end', {})

        s = start.get(list(start.keys())[0], '')  # Default to empty string if start is missing
        s_yr = s[:5] if s.startswith('-') else s[:4]  # Extract year part

        e = end.get(list(end.keys())[0], s)  # Default to start year if end is missing or empty
        e_yr = e[:5] if e.startswith('-') else e[:4]  # Extract year part

        result.append([int(s_yr), int(e_yr)])

    return result

Explanation of Changes:

Safe Access: Uses get() method to safely retrieve values from start and end dictionaries without causing KeyError if keys are missing.
Default Values: Handles cases where 'end' is not present or empty by defaulting e to s, ensuring that the function does not break unexpectedly.
String Manipulation: Adjusts how years (s_yr and e_yr) are extracted to ensure compatibility with varying formats (YYYY and -YYYY).

These adjustments should improve the robustness of timespansReduce function when processing complex data structures like the example you provided. Adjust further based on additional complexities or specific requirements of your application.

Re-think of Dataset Upload & Validation

[ ] Remove options for default fclasses and timespans.
[ ] For the simplest possible UX, consider removing all other optional (metadata) inputs (these values can be added later, before publication).

Robustness & Maintainability: Data Formats, Validation, & Insertion
The current code for validation and database-insertion of JSON datasets spans 118 lines; by contrast, that for delimited text datasets (ds_insert_delim and its helper functions) spans 475 lines and is commensurately more difficult to maintain and extend.
The pandas library (already in use) indicates a way to simplify not only the codebase but also the preparation of datasets, with its json_normalize method. Using it with recursion into nested structures, LPF JSON can be converted to delimited text reversibly and without any loss of data. The entire LPF JSON feature example given at https://github.com/LinkedPasts/linked-places-format could, for example, could be transformed into the following 80 flattened columns:
type | @id | properties.title | properties.ccodes.0 | properties.fclasses.0 | when.timespans.0.start.in | when.timespans.0.end.in | when.periods.0.name | when.periods.0.@id | when.periods.1.name | when.periods.1.@id | when.label | when.duration | when.certainty | names.0.toponym | names.0.lang | names.0.citations.0.label | names.0.citations.0.year | names.0.citations.0.@id | names.0.when.timespans.0.start.in | names.1.toponym | names.1.lang | names.1.when.timespans.0.start.in | names.1.when.certainty | types.0.identifier | types.0.label | types.0.sourceLabels.0.label | types.0.sourceLabels.0.lang | types.0.when.timespans.0.start.in | geometry.type | geometry.geometries.0.type | geometry.geometries.0.coordinates.0 | geometry.geometries.0.coordinates.1 | geometry.geometries.0.when.timespans.0.start.in | geometry.geometries.0.when.timespans.0.end.in | geometry.geometries.0.citations.0.label | geometry.geometries.0.citations.0.@id | geometry.geometries.0.certainty | geometry.geometries.1.type | geometry.geometries.1.coordinates.0 | geometry.geometries.1.coordinates.1 | geometry.geometries.1.geowkt | geometry.geometries.1.when.timespans.0.start.in | geometry.geometries.1.certainty | links.0.type | links.0.identifier | links.1.type | links.1.identifier | links.2.type | links.2.identifier | links.3.type | links.3.identifier | links.4.type | links.4.identifier | links.5.type | links.5.identifier | relations.0.relationType | relations.0.relationTo | relations.0.label | relations.0.when.timespans.0.start.in | relations.0.when.timespans.0.end.in | relations.1.relationType | relations.1.relationTo | relations.1.label | relations.1.when.timespans.0.start.in | relations.2.relationType | relations.2.relationTo | relations.2.label | relations.2.when.timespans.0.start.in | relations.2.citations.0.label | relations.2.citations.0.year | relations.2.citations.0.@id | relations.2.certainty | descriptions.0.@id | descriptions.0.value | descriptions.0.lang | depictions.0.@id | depictions.0.title | depictions.0.license

The format is extensible for larger arrays (but unlike LPF JSON does not accommodate metadata, which would need to be provided separately, as is already the case for LP-TSV). Most datasets would probably use considerably fewer columns, and any delimited text which uses this column-naming convention could be converted to LPF JSON. WHG users could submit flattened LPF (f-LPF?) as an alternative to LP-TSV.
The processing pipeline for an uploaded dataset could then be simplified. Chunking and streaming must be used where indicated in order to accommodate the possibility of very large files which would compromise memory and performance of the server.

Try loading a line (csv, tsv) or chunk (xlsx, ods) in a dataframe: is it delimited text? If not, go to 4.
Is it LP-TSV? If so, rename columns to conform to f-LPF (e.g.start -> when.timespans.0.start.in) before continuing...
Is it f-LPF? If so, either line-by-line or by chunking, convert and write to LPF (JSON) before continuing...
Using chunking, try loading as JSON and coincidentally check encoding: is it UTF-8 JSON? If not, exit.
Validate using streaming,
Insert into database with ds_insert_json.

Such a pipeline would readily allow for any extension of the LPF standard, such as the requirement for fclasses in default of types, by a modification of the JSON schema (and perhaps of any related Django data models). The fclasses need a definition, and a "oneOf" enforcement in conjunction with types.
The current JSON schema (datasets/static/validate/schema_lpf_v1.2.2.json) could benefit from the use of definitions for common elements like timespans, periods, and when. Additionally (and based on a comparison with the example LPF JSON), ChatGPT identifies several errors that it could easily address (subject, of course, to checking):

Inconsistent or Missing Type Declarations: Some properties do not have type declarations, which could lead to issues when validating against the schema.
Property Naming and Structure: The properties' names and structure do not fully align with the example JSON, especially in nested objects.
Incorrect or Missing Required Properties: Certain required properties are missing in the schema, such as the title in the properties object and the type in the geometry object.
Pattern Properties: Some pattern properties are not specified correctly or are not necessary.

Further improvements could be made in the validation pipeline with use of jsonschema CustomValidators, to check for example that end dates are no earlier than start dates.

See also #350 re polygon validation/correction

Draft LPF v2.0 JSON Schema
Based on v1.2.2 and on example record here. Uses definitions where appropriate to avoid duplication and improve flexibility.
Further improvement:

[ ] Standardization of Titles: Ensure consistent titling across all properties and definitions. For example, "Types->Items->Label" vs. "The Type Schema." Consistent naming conventions should be used throughout.
[ ] Pattern Validation: Some string patterns are very loose, e.g., ^(.*)$. Consider using more specific regex patterns where possible to improve data validation.
[ ] Required Fields: Ensure that required fields are consistently applied across similar structures. For example, some object definitions have no required fields, while others do.
[ ] Object Descriptions: Adding descriptions for more properties could improve schema readability and usability. It can provide more context for users implementing the schema.
[ ] Examples in Schema: Some properties have examples provided, while others do not. Adding examples for all relevant fields could enhance clarity.
[ ] Default Values: Consider setting meaningful default values where possible, instead of leaving them empty. This can help with data consistency.
[ ] Enum Definitions: For properties using enum, ensure that all possible values are listed and adequately described in the schema documentation.
[ ] Array Definitions: Ensure arrays have clear minimum and maximum item constraints where applicable, to enforce expected data ranges.
[ ] Required Fields in Arrays: In some array objects, required fields are not consistently defined. For instance, in types and names, ensure that all items within arrays have necessary required fields.
[ ] Documentation: Ensure that there is enough documentation or inline comments in the schema explaining complex definitions or structures.
[ ] Consistency in Property IDs: Ensure all $id fields are consistently applied and match the property path. Some $id fields might be unnecessary and could be removed.
[ ] Schema Definitions: Consider centralizing common properties or object structures into definitions to avoid redundancy and make the schema easier to maintain.
[ ] Property Names: Review property names for clarity and consistency, particularly for nested properties. Avoid overly generic names where possible.
[ ] Additional Properties: Review the use of "additionalProperties": true to ensure that it's only used where flexibility is needed and does not introduce unintended flexibility.
[ ] Add CSL JSON Citation root property

{
    "$schema": "http://json-schema.org/draft-07/schema#",
    "$id": "schema_lpf_v2.0.json",
    "type": "FeatureCollection",
    "@context": "https://raw.githubusercontent.com/LinkedPasts/linked-places/master/linkedplaces-context-v1.1.jsonld",
    "title": "The Linked Places Schema",
    "additionalProperties": false,
    "required": [
        "@id",
        "type",
        "properties",
        "names"
    ],
    "definitions": {
        "when": {
            "type": "object",
            "properties": {
                "timespans": {
                    "type": "array",
                    "items": {
                        "$ref": "#/definitions/timespan"
                    }
                },
                "periods": {
                    "type": "array",
                    "title": "When->Periods",
                    "items": {
                        "$ref": "#/definitions/period"
                    }
                },
                "label": {
                    "type": "string",
                    "title": "When->Label"
                },
                "duration": {
                    "$ref": "#/definitions/duration"
                }
            }
        },
        "timespan": {
            "type": "object",
            "properties": {
                "start": {
                    "type": "string",
                    "format": "date",
                    "title": "Timespan->Start"
                },
                "end": {
                    "type": "string",
                    "format": "date",
                    "title": "Timespan->End"
                }
            },
            "required": [
                "start",
                "end"
            ]
        },
        "period": {
            "type": "object",
            "properties": {
                "name": {
                    "type": "string"
                },
                "uri": {
                    "type": "string"
                }
            },
            "required": [
                "label"
            ]
        },
        "duration": {
            "type": "string",
            "title": "Duration",
            "examples": [
                "P1Y",
                "P2M"
            ],
            "pattern": "^[P][0-9]{1,2}[Y|M|D]$"
        },
        "citations": {
            "type": "array",
            "title": "Citations",
            "items": {
                "$ref": "#/definitions/citation"
            }
        },
        "citation": {
            "type": "object",
            "title": "Citation",
            "properties": {
                "@id": {
                    "type": [
                        "string",
                        "null"
                    ],
                    "title": "Citation->@id",
                    "examples": [
                        "http://foofah.org"
                    ]
                },
                "label": {
                    "type": "string",
                    "title": "Citation->Label",
                    "examples": [
                        "Blow, J. (1708)"
                    ],
                    "pattern": "^(.*)$"
                }
            }
        },
        "lang": {
            "type": "string",
            "title": "Language",
            "examples": [
                "en",
                "fr"
            ],
            "pattern": "^[a-z]{2}$"
        },
        "certainty": {
            "type": "string",
            "title": "Certainty",
            "enum": [
                "certain",
                "less-certain",
                "uncertain"
            ]
        }
    },
    "properties": {
        "type": {
            "$id": "#/properties/type",
            "type": "string",
            "title": "The Type Schema",
            "enum": [
                "Feature"
            ]
        },
        "@id": {
            "$id": "#/properties/@id",
            "type": "string",
            "title": "The @id Schema",
            "default": "",
            "examples": [
                "https://foofah.com"
            ],
            "pattern": "^(.*)$"
        },
        "properties": {
            "$id": "#/properties/properties",
            "type": "object",
            "title": "The Properties Schema",
            "required": [
                "title"
            ],
            "properties": {
                "title": {
                    "$id": "#/properties/properties/properties/title",
                    "type": "string",
                    "title": "Properties->Title",
                    "default": "",
                    "examples": [
                        "San Miguel"
                    ],
                    "pattern": "^(.*)$"
                },
                "ccodes": {
                    "$id": "#/properties/properties/properties/ccodes",
                    "type": "array",
                    "title": "Properties->Ccodes",
                    "items": {
                        "$id": "#/properties/properties/properties/ccodes/items",
                        "type": "string",
                        "title": "Properties->Ccodes->Items",
                        "default": "",
                        "examples": [
                            "CR"
                        ],
                        "pattern": "^([a-zA-Z]{2})$"
                    }
                },
                "fclasses": {
                    "type": "array",
                    "title": "TODO",
                    "items": {
                        "type": "string"
                    }
                }
            }
        },
        "names": {
            "$id": "#/properties/names",
            "type": "array",
            "title": "The Names Schema",
            "items": {
                "$id": "#/properties/names/items",
                "type": "object",
                "title": "Names->Items",
                "properties": {
                    "toponym": {
                        "$id": "#/properties/names/items/properties/toponym",
                        "type": "string",
                        "title": "Names->Items->Toponym",
                        "examples": [
                            "San Miguel"
                        ],
                        "pattern": "^(.*)$"
                    },
                    "ethnonym": {
                        "$id": "#/properties/names/items/properties/ethnonym",
                        "type": "string",
                        "title": "Names->Items->Ethnonym",
                        "examples": [
                            "Lebenians"
                        ],
                        "pattern": "^(.*)$"
                    },
                    "lang": {
                        "$ref": "#/definitions/lang"
                    },
                    "when": {
                        "$ref": "#/definitions/when"
                    },
                    "citations": {
                        "$ref": "#/definitions/citations"
                    }
                },
                "oneOf": [
                    {
                        "required": [
                            "toponym"
                        ]
                    },
                    {
                        "required": [
                            "ethnonym"
                        ]
                    }
                ],
                "additionalProperties": true
            }
        },
        "types": {
            "$id": "#/properties/types",
            "type": "array",
            "title": "The Types Schema",
            "items": {
                "$id": "#/properties/types/items",
                "type": "object",
                "title": "Types->Items",
                "properties": {
                    "label": {
                        "$id": "#/properties/types/items/properties/label",
                        "type": "string",
                        "title": "Types->Items->Label",
                        "default": "",
                        "examples": [
                            "village"
                        ],
                        "pattern": "^(.*)$"
                    },
                    "identifier": {
                        "$id": "#/properties/types/items/properties/identifier",
                        "type": "string",
                        "title": "Types->Items->Identifier",
                        "default": "",
                        "examples": [
                            "aat:300008372"
                        ],
                        "pattern": "^(.*)$"
                    },
                    "sourceLabels": {
                        "type": "array",
                        "title": "Source Labels",
                        "items": {
                            "type": "object",
                            "properties": {
                                "label": {
                                    "type": "string",
                                    "title": "Source Labels->Label",
                                    "examples": [
                                        "Pueblo"
                                    ],
                                    "pattern": "^(.*)$"
                                },
                                "lang": {
                                    "$ref": "#/definitions/lang"
                                }
                            },
                            "required": [
                                "label"
                            ]
                        }
                    },
                    "when": {
                        "$ref": "#/definitions/when"
                    }
                }
            }
        },
        "geometry": {
            "$id": "#/properties/geometry",
            "type": "object",
            "title": "The Geometry Schema",
            "required": [
                "type"
            ],
            "properties": {
                "type": {
                    "$id": "#/properties/geometry/properties/type",
                    "type": "string",
                    "enum": [
                        "GeometryCollection",
                        "Point",
                        "MultiPoint",
                        "LineString",
                        "MultiLineString",
                        "Polygon",
                        "MultiPolygon"
                    ],
                    "title": "Geometry->Type"
                },
                "geometries": {
                    "$id": "#/properties/geometry/properties/geometries",
                    "type": "array",
                    "title": "Geometries",
                    "items": {
                        "$id": "#/properties/geometry/properties/geometries/items",
                        "type": "object",
                        "title": "Geometries->Items",
                        "properties": {
                            "type": {
                                "$id": "#/properties/geometry/properties/geometries/items/properties/type",
                                "type": "string",
                                "title": "Geometries->Items->Type",
                                "enum": [
                                    "Point",
                                    "MultiPoint",
                                    "LineString",
                                    "MultiLineString",
                                    "Polygon",
                                    "MultiPolygon"
                                ]
                            },
                            "coordinates": {
                                "type": "array",
                                "items": {
                                    "type": "number"
                                }
                            },
                            "geowkt": {
                                "$id": "#/properties/geometry/properties/geometries/items/properties/geowkt",
                                "type": "string",
                                "title": "Geometries->Items->Geowkt",
                                "pattern": "^(.*)$"
                            },
                            "when": {
                                "$ref": "#/definitions/when"
                            },
                            "citations": {
                                "$ref": "#/definitions/citations"
                            },
                            "certainty": {
                                "$ref": "#/definitions/certainty"
                            }
                        }
                    }
                }
            }
        },
        "relations": {
            "$id": "#/properties/relations",
            "type": "array",
            "title": "The Relations Schema",
            "items": {
                "$id": "#/properties/relations/items",
                "type": "object",
                "title": "Relations->Items",
                "required": [
                    "relationType",
                    "relationTo",
                    "label"
                ],
                "properties": {
                    "relationType": {
                        "type": "string",
                        "examples": [
                            "gvp:broaderPartitive"
                        ],
                        "pattern": "^(.*)$"
                    },
                    "relationTo": {
                        "type": "string",
                        "pattern": "^(.*)$"
                    },
                    "label": {
                        "type": "string",
                        "pattern": "^(.*)$"
                    },
                    "when": {
                        "$ref": "#/definitions/when"
                    },
                    "citations": {
                        "$ref": "#/definitions/citations"
                    },
                    "certainty": {
                        "$ref": "#/definitions/certainty"
                    }
                }
            }
        },
        "links": {
            "$id": "#/properties/links",
            "type": "array",
            "title": "The Links Schema",
            "items": {
                "$id": "#/properties/links/items",
                "type": "object",
                "title": "Links->Items",
                "properties": {
                    "type": {
                        "type": "string",
                        "enum": [
                            "closeMatch",
                            "exactMatch",
                            "primaryTopicOf",
                            "subjectOf",
                            "seeAlso"
                        ]
                    },
                    "identifier": {
                        "type": "string"
                    }
                }
            }
        },
        "when": {
            "$ref": "#/definitions/when"
        },
        "descriptions": {
            "$id": "#/properties/descriptions",
            "type": "array",
            "title": "The Descriptions Schema",
            "items": {
                "$id": "#/properties/descriptions/items",
                "type": "object",
                "title": "Descriptions->Items",
                "required": [
                    "value"
                ],
                "properties": {
                    "@id": {
                        "$id": "#/properties/descriptions/items/properties/@id",
                        "type": "string",
                        "title": "Descriptions->Items->@id",
                        "examples": [
                            "https://en.wikipedia.org/wiki/Abingdon-on-Thames"
                        ],
                        "pattern": "^(.*)$"
                    },
                    "value": {
                        "$id": "#/properties/descriptions/items/properties/value",
                        "type": "string",
                        "title": "Descriptions->Items->Value",
                        "examples": [
                            "...a historic market town and civil parish..."
                        ],
                        "pattern": "^(.*)$"
                    },
                    "lang": {
                        "$ref": "#/definitions/lang"
                    }
                }
            }
        },
        "depictions": {
            "$id": "#/properties/depictions",
            "type": "array",
            "title": "The Depictions Schema",
            "items": {
                "$id": "#/properties/depictions/items",
                "type": "object",
                "title": "Depictions->Items",
                "required": [
                    "@id",
                    "title",
                    "license"
                ],
                "properties": {
                    "@id": {
                        "$id": "#/properties/depictions/items/properties/@id",
                        "type": "string",
                        "title": "Depictions->Items->@id",
                        "examples": [
                            "https://commons.wikimedia.org/wiki/File:ThamesAtAbingdon.jpg"
                        ],
                        "pattern": "^(.*)$"
                    },
                    "title": {
                        "$id": "#/properties/depictions/items/properties/title",
                        "type": "string",
                        "title": "Depictions->Items->Title",
                        "default": "",
                        "examples": [
                            "The River Thames at Abingdon, Oxfordshire"
                        ],
                        "pattern": "^(.*)$"
                    },
                    "license": {
                        "$id": "#/properties/depictions/items/properties/license",
                        "type": "string",
                        "title": "Depictions->Items->License",
                        "default": "",
                        "examples": [
                            "cc:by-sa/3.0/"
                        ],
                        "pattern": "^(.*)$"
                    }
                }
            }
        }
    }
}

FWIW, there is a new (9 Aug) draft lpo: ontology.
Also a corresponding (I think) context file for LPF
Not sure the context matters much as I don't think anyone attempts to do reasoning against LPF as an RDF syntax
These were developed in the course of making a .ttl export of WHG place data as an experiment.

I'm incorporating some of the compaction functionality of pyld for preprocessing uploads, based on the context; we could easily offer downloads that employ the expansion functionality for anyone who wants it.

I'm incorporating some of the compaction functionality of pyld for preprocessing uploads, based on the context; we could easily offer downloads that employ the expansion functionality for anyone who wants it.

I am not prepared to validate the draft @context and ontology, and expansion is not something anyone has ever indicated an interest in. There absolutely would be issues and gotchas, and I will not troubleshoot them nor adjust my drafts.

I think json-ld expansion to rdf/xml is very low priority.
Some context:
The new draft ontology and context are part of an experimental .ttl serialization, the script for which will be a PR before long. There would be a one-shot off-hours export of all public Place records in some detail, imported to a GraphDB database on another server, where some experimental UI work would happen.
The script could be modified in the future to output updates of newly public records for addition to the graph. But that is pending a finding that it is useful in some way. This graph experiment may expand to adding records from other sources.
The impact on WHG for the time being (and quite some time) will be the addition of a single management command that will be accessed only manually - for initial testing on limited number of records, and then one large export.
The exported .ttl is not fully LP format-compatible

Thanks - json-ld expansion is not even low priority.
It strikes me that because the staging server is running with an exact copy (made a few days ago using this script) of the main database, this kind of thing might best be done there?


Support improve App Theme settings

To make theme settings clearer, user friendly and with no hidden assumptions, this PR adds new way to set theme settings.
Previously default and hidden setting was Automatic. Once use change that setting they could never go back to Auto and was either Light or Dark.
Now users can explicitly set Automatic/Light/Dark for full flexibility.
Dark Amoled mode is now always available and if is set with Auto, it still triggers Dark Amoled when system is in Dark.

Merged in #163


Allow a custom JSON parser to be used

In our setup we need to use json-bigint to handle Big Integers without losing precision. This is not currently supported by json-parse-async.
This (fairly trivial) pull request adds an extra argument to the exposed parseJSON method to specify the parse method to use (defaulting to JSON.parse), like this:
const parseJson = require('json-parse-async')
const JSONbig = require('json-bigint')

// As promise:
parseJson(json, null, JSONbig.parse).then(...)

// Or, using a callback:
parseJson(json, cb, JSONbig.parse)

Tests have been included. I'm new to the world of CoffeeScript so feel free to point out if there are issues with my test method or anything else.

Thanks for this PR @bverhoeven.
I felt this library os outdated, I created a swiss knife library to use with JSON methods called json-future.
Can you check if jsonFuture#parse API is great for your user case?


Soletta Module: Refactor machine_learning module.

Some methods for sync and non sync versions of the module
are the same with minor differences. This patch creates a
unified implementation for these functions.
Signed-off-by: Guilherme Iscaro<EMAIL_ADDRESS>

@bdilly Could you please review this patch?

@cabelitos yeah, that was the idea. Seems good. I didn't test it.

I tested it, it works like a charm :)


Different types being returned from a single cmdlet

When generating cmdlets for AppService, we have the following two variants:

Variant
Output Type
Operation ID

GetAzAppServicePlanWebApp_List
string
AppServicePlans_ListWebAppsByHybridConnection

GetAzAppServicePlanWebApp_List1
Microsoft.Azure.PowerShell.Cmdlets.WebSite.Models.Api20180201.ISite
AppServicePlans_ListWebApps

As a result of this, we generate a cmdlet, Get-AzAppServicePlanWebApp, with the following OutputType:
[OutputType('System.String', 'Microsoft.Azure.PowerShell.Cmdlets.WebSite.Models.Api20180201.ISite')]

How should we handle these cases where multiple types are being returned from a single cmdlet call due to how we're consolidating variants? Ideally, a cmdlet would return a single type (excluding collections and sub/super types) no matter the parameter set used.

Closing as this can be resolved with a directive that renames or removes the variant returning a string.


500 error with redmine 3.4.2 when clicking existing issue

Hi,
after updating redmine from 3.3.3 to 3.4.2 we get this error when clicking on an existing error.
Our installation is based on the latest bitnami stack (Redmine V3.3.3 updated to V3.4.2).
At the old installation (V3.3.3) the plugin runs next to several plugins without failure.
The error is independend from other plugins (for testing we only installed this plugin => error occurs)
Production log says "ActionView::Template::Error (undefined method `render_custom_fields_rows'"
Any ideas on how we can fix it are highly appreciated.
Here is the part from production.log:
Started GET "/redmine/issues/4559" for <IP_ADDRESS> at 2017-07-31 17:03:06 +0200
Processing by IssuesController#show as HTML
  Parameters: {"id"=>"4559"}
  Current user: admin (id=1)
  Rendered issues/_action_menu.html.erb (9.8ms)
  Rendered plugins/event_notifications/app/views/issues/show.html.erb within layouts/base (176.8ms)
Completed 500 Internal Server Error in 525ms (ActiveRecord: 91.6ms)

ActionView::Template::Error (undefined method `render_custom_fields_rows' for #<#<Class:0x0055e7a98425$
Did you mean?  render_custom_fields_tabs
               render_custom_field_values):
    68:     end
    69:   end
    70: end %>
    71: <%= render_custom_fields_rows(@issue) %>
    72: <%= call_hook(:view_issues_show_details_bottom, :issue => @issue) %>
    73: </div>
    74:
  plugins/event_notifications/app/views/issues/show.html.erb:71:in `_plugins_event_notifications_app_v$
  app/controllers/issues_controller.rb:106:in `block (2 levels) in show'
  app/controllers/issues_controller.rb:99:in `show'
  lib/redmine/sudo_mode.rb:63:in `sudo_mode'

Same issue here using Redmine 3.4.2.stable.
Thanks,
Michele

Same issue here, same constellation.
Thanks for fixing the problem.

This problem with view is not one. Additional problem is witch saving settings of plugin. On redmine 3.4.x are some changes, that can old plugins paralized.
Is this plugin still developed?

Hey jrupesh,
same issue here using Redmine 3.4.3 stable. Are you developing the plugin futher? Thanks for a quick reaction.

Issue corrected with latest commit.
Thanks.

sorry, which commit? The last one here seems to be on the Jan 20, 2017...
Thanks a lot,
Michele

Sorry mdima,
Missed to push to master branch. The latest one is available now. (Also in 3.0.0 branch)

Hi jrupesh, thanks a lot! I downloaded the latest version, installed, checked, everything seems to work, thanks a lot!!
Michele


Change apiVersion to zalando.org/v1

This changes the apiVersion defined for the AWSIAMRole CRD from amazonaws.com/v1 to zalando.org/v1.
This is unfortunately a breaking change that we feel the need to make as we are preparing to roll this out in Zalando.
The reason to make this change is to ensure that Zalando as a company is not breaking any Trademark laws. While this project has been developed by me until now. As I'm an employee of Zalando because of European Copyright laws the copyright of the project falls to Zalando as this is related to my daily work. Therefore it makes sense to use zalando.org/v1 instead of coming up with another name which may be trademarked by someone else.
@seh already brought up this issue in the original PR.
Migration steps

Apply new CRD:
kubectl apply -f docs/aws_iam_role_crd.yaml

Copy old resources to new version:
#!/bin/bash

for ns in $(kubectl get ns --no-headers -o "custom-columns=:metadata.name"); do
    for res in $(kubectl --namespace "$ns" get awsiamroles.amazonaws.com --no-headers -o custom-columns=:metadata.name); do
        kubectl --namespace "$ns" get awsiamroles.amazonaws.com "$res" -o json | sed 's#amazonaws.com/v1#zalando.org/v1#' | kubectl apply -f -
    done
done

Deploy new version of controller >=0.1.0 (See https://github.com/mikkeloscar/kube-aws-iam-controller#setup).

Delete old resources and old CRD:
kubectl delete crd awsiamroles.amazonaws.com

Pull Request Test Coverage Report for Build 112

4 of 4 (100.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 67.454%

Totals

Change from base Build 109:
0.0%

Covered Lines:
514

Relevant Lines:
762

💛  - Coveralls

Given the version bump, do you have any plans to phase out the older IAM role binding technique of mounting Secrets named with a special prefix? I'd prefer to be able to deploy the controller at least with the pod cache disabled. If we can't remove it for fear of breaking existing users, perhaps we could disable it via command-line flag.
I was hoping that the next major version bump would be our chance to remove that feature.

I created #25 to track this. I agree we should phase it out.

@mikkeloscar if your deployment is with rbac you must also migrate the clusterrole

@Shinzu thanks, added this as a step in the description above!


Portfolio_API

add function:
get_portfolio
show all portfolios' name, amount, and currency_id
add_portfolio
give portfolio name, amount, and currency_id to create a new portfolio
remove_portfolio
give portfolio name to remove that portfolio
change_portfolio
give portfolio name and new amount to change that portfolio

@w89652 I have solved some conflicts, please check the correctness and pull.

@w89652 After modifying the requested change, please resolve the conflicts.


Remove support for pfr<=0.9.x

This draft aims to move us towards cutting support for the older to_beam recipe version. This ought to simplify developer work and cut down on CI complexity. #144

@cisaacstern It occurs to me you might have an opinion here. I'm for pulling the trigger and moving things forward as i think there are further issues with the ways recipes are handled that will require potential further deprecation or at least heavy guidance on the semantics of the two approaches - PTransform construction vs lazily building our full pipeline inside a function that will serve as the recipe (this is enabled by the PR about lazy pipeline construction #168)

Thanks for the ping I will take a look this afternoon.

well, codecov was complaining b/c the number of removals seemed to reduce the overall ratio of coverage, not b/c things changed too much. I added some codecov.yml rules that exclude /tests/ and obvious things that the recipes repo excluded and we're gravy 🥳
gonna merge this sucker

Awesome! Thanks all!


Don't animate sidebar unless transitioning a page?

There are a few scenarios where you will see the Sidebar animate out unexpectedly.  Most notable is in Firefox Android when opening odysee.com (maybe need Incognito) ...  I see the Sidebar flash and slowly animate out.
I think the animation is only needed for:

Explicitly changing sidebar state from pinned to minimized
From a page with sidebar pinned, enter a File Page (where sidebar cannot be pinned).

Hey @MaxKotlan, maybe this one would be up your sleeve. I notice this most on Firefox android also. No rush though!

Sure, I'd be happy to take a look at that. I can reproduce the issue on mobile too. I'll see what I can do to fix it.


XLIFF unit name and ID can't be shown?

Hi there,
We've been using XLIFF 2.0 for our translation exchange format, and our game engine (Unity) spits out an eg. GameTables_de.xlf file that look like
<?xml version="1.0" encoding="utf-8"?>
<xliff xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="2.0" srcLang="en" trgLang="de" xmlns="urn:oasis:names:tc:xliff:document:2.0">
  <file id="025803cd740852c4bba2614cf5c1f8e4" original="Assets/Game/Data/Localization/Tables/UI/UI_en.asset">
    <group id="f842d7241a1c0cd45adfe335a3afd50a" name="UI">
      <unit id="86546407325106176" name="Leaderboard_RetryRequestButton" xml:space="preserve">
        <notes>
          <note>If the leaderboards fail to be retrieved, the player can press this "retry" button to try connecting and retrieving leaderboard data again.</note>
        </notes>
        <segment state="translated">
          <source>Retry</source>
          <target>Wiederholen</target>
        </segment>
      </unit>
      ...
    </group>
  </file>
</xliff>

The game engine's localization system assigns a unique number (guid) for each string, which it then puts into the id attribute of the unit, but then includes the user-facing "key" we enter as the name attribute. Each string table in the game is given its own group element, with the table title in the name attribute. The above is an entry with key "Leaderboard_RetryRequestButton" from our "UI" table.
As far as I can tell there isn't a way to add additional columns to POEdit to display the name or id attribute?
I found a checkbox to show the "id", but it seems to just print out more of an "index within the POEdit list".

It does seem to show the unit's id attribute in the details panel when you have a row focused, along with the comment:

But it'd be give translators much more context to also be able to see the key we entered.
Ideally POEdit could also have an option to group things in the list, almost like a tree-view where you can expand/contract groups. Since we might have a table (that's exported as a <group> element) for general UI, one for Credits, one for Dialogue, one for GameModes, one for OptionsMenu etc. and leave the group name out of the individual string keys. So exposing the group to the translators (even if just as another column instead of hierarchically) would probably also make their lives easier.
Thanks!

GameTables_de.xlf file that look like

Is this accurate, i.e. there's no additional metadata? Would you be able to share (privately<EMAIL_ADDRESS>real-life file for better testing?
You're correct in your observations as well as in this:

Ideally POEdit could also have an option to group things in the list, almost like a tree-view where you can expand/contract groups.

This is indeed the plan, together with exposing the symbolic IDs in the UI as actual IDs. As you can probably tell, there are UI artifacts from Poedit being originally for PO files only.
Doing that is a bit more work than I won't be able to fit into 3.1, but showing the symbol ID in the Notes for translators sidebar is simple enough to make it, with further improvements following.

Thanks for the response!
Yeah it seems like there's a <file> per string table, with a single <group> within each <file>

Which might not be ideal if the UI did expose it as an expandable tree cause you'd have a redundant group node under each file node unless POEdit did some kind of smart "flattening"?
Here's a trimmed down XLF with just that single string in it if it's useful for testing? (had to zip it for GitHub to let me upload it)
UnitySampleLocExport.xlf.zip
Or if you need more strings let me know and we can maybe email it privately. Or feel free to generate your own string tables via the Unity localization package if wanting to sift through the details of their output format.
It turns out CrowdIn also don't show the name attribute anywhere, so I suspect Unity's breaking convention a bit in how they chose to put the machine-readable ID in the id attribute and the human-readable separately.

Or if you need more strings let me know and we can maybe email it privately.

It's always useful to have "real-life" files / sets of files to have a better understanding e.g. which parts of spec are actually used (full XLIFF 2.x is enormous spec and approximately nobody uses all of it…).

It turns out CrowdIn also don't show the name attribute anywhere, so I suspect Unity's breaking convention a bit in how they chose to put the machine-readable ID in the id attribute and the human-readable separately.

This was poor handling on Poedit's end, Unity is doing the Right Thing ™ here. Quoting the spec on the meaning of the optional name attribute:

Resource name - the original identifier of the resource […] For example: the key in the key/value pair in a Java properties file, the ID of a string in a Windows string table, the index value of an entry in a database table, etc

…and so in 3.1, after 4c4fef37068705e4418b6a68a2f52c64938dbaa8, Poedit will prefer to use (res)name for the ID if present.


Ci 159 roatp api .net client consumer contracts tests

Since this is a public API Pact can't be used to it's fullest so probably isn't the best solution for testing the API


PR possible ?

add c4ei possible ?

You didn't reply to my message in the previous PR.
Your network does not follow the requirements:
https://support.onout.org/hc/1331700057/32/addnewnetwork?category_id=8

Oh i see i will check thank you very much
2023년 8월 3일 (목) 오후 3:06, NotEternal @.***>님이 작성:

You didn't reply to my message in the previous PR.
Your network does not follow the requirements:
https://support.onout.org/hc/1331700057/32/addnewnetwork?category_id=8
—
Reply to this email directly, view it on GitHub
https://github.com/noxonsu/unifactory/pull/174#issuecomment-1663342619,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXPZLKP4LTJLMWPVYJBU2A3XTM5W5ANCNFSM6AAAAAA3CCAVC4
.
You are receiving this because you authored the thread.Message ID:
@.***>


Surprising behaviour of from_raw_parts

I'm trying to write code that passes a pointer correctly via FFI from Rust -> C -> Rust, while respecting strict provenance. I wrote this code which I would assume preserves the provenance information or at-least triggers my assert that it can't store the provenance information in a u64.
#[derive(PartialEq, Eq, Debug, Clone)]
#[repr(C)]
struct CompCount {
    val: i32,
    comp_count: Cell<u32>,
}

fn main() {
    type CompCountRawPartsMut = (*mut (), <CompCount as ptr::Pointee>::Metadata);
    type CompCountRawParts = (*mut (), <CompCount as ptr::Pointee>::Metadata);

    let self_ptr: CompCountRawPartsMut =
        Box::into_raw(Box::new(CompCount::new(3))).to_raw_parts();

    assert!(mem::size_of::<CompCountRawPartsMut>() == mem::size_of::<u64>());
    // Over-aligning is ok.
    assert!(mem::align_of::<CompCountRawPartsMut>() <= mem::align_of::<u64>());

    let val: u64 = unsafe { mem::transmute::<CompCountRawPartsMut, u64>(self_ptr) };

    let (addr, metadata) = unsafe { mem::transmute::<u64, CompCountRawParts>(val) };
    let self_ptr_read: *const CompCount = ptr::from_raw_parts(addr, metadata);

    let x: &CompCount = unsafe { &*self_ptr_read };
    dbg!(x);

    // drop code omitted for brevity.
}

When I run this, I get: is a dangling pointer (it has no provenance)

I guess val is a number and has now lost any meaning as pointer, but I was still surprised by the effect. I had assumed that to_raw_parts gives me all the necessary information including provenance in the metadata. But probably that's the wrong mental model, can you even serialize provenance?

You aren't respecting strict provenance. The whole idea of strict provenance is to not store pointers as integers, because round-tripping pointers through integers has complicated semantics which we aren't completely sure of yet.
What we are basically totally sure of is that you cannot round-trip via transmutes, which is what you are trying to do here. The reason for this is that there are many ways to transmute in Rust that do not involve calling the function, so a model that permits this adds a lot of complexity to all loads of integer type.
If you need to round-trip, use casts. The documentation for all the core strict provenance APIs may be helpful, consult ptr::addr, ptr::invalid, ptr::expose_addr, and ptr::from_exposed_addr. The former are transmutes and the latter are currently implemented with as casts. It sounds to me like you need expose semantics as a result of a C API smuggling a pointer type through an integer type API.

can you even serialize provenance?

No, you cannot -- provenance is purely a 'ghost' component of the Abstract Machine that flows with each pointer, and disappears when the pointer is turned into an integer. Provenance cannot be attached to anything but pointers (and entirely untyped data, aka MaybeUninit<T>).

Thanks for the clarification, I think I was thinking too much in terms of CHERI where provenance is some concrete information in the form of bits. As a hint if you aren't aware of this already, I found the documentation surrounding this topic to be rather confusing, even as someone who thought of himself as familiar with the concept.

Yeah, sadly we don't have good docs on this yet because we are still in the process of slowly nudging Rust into accepting that provenance is a thing. ;)  Things that are officially documented are basically stable promises, and the lang team is not yet ready to make stable promises on these details of the provenance model.
 I think Miri implements the model of provenance we will eventually want to have, but I cannot promise that.

Did you read the documentation for ptr::addr? It is intended to explain that you cannot do what you were trying to do. If it doesn't, can you advise on any changes?


[haskellUpdates] Fix reflex, reflex-dom and dependencies.

Motivation for this change
Would be really nice to be able to build reflex packages with nixpkgs without needing to rely on https://github.com/reflex-frp/reflex-platform in all cases.
Todo List

[ ] Discuss adding jailbreakConditional conditional dependencies to cabal-jailbreak
[x] Create upstream issues about version bounds
[x] Make PR for patch work with newer dependent-sum
[x] Create upstream issues about old test dependencies
[x] Figure out why reflex-dom-core remeins marked as broken

Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[x] Fits CONTRIBUTING.md.

I need help on this one. @peti I have run hackage2nix and reflex-dom-core is not being mentioned at all in configuration-hackage2nix.yaml. But still there is a broken flag generated in the hackage-packages.nix expression for reflex-dom-core. Any hints?

reflex-dom-core is considered broken, because it has a dependency on the non-existent Haskell package chrome-test-utils. Note the explicit null assignment that's generated at the end of the expression.

Ah, thank you @peti. Well that wont be a problem, if we disable the tests.

btw. can I do something to get a Haskell label on my PR?

I don‘t feel like I have anything to do with the build error @GrahamcOfBorg is complaining about.
The build works just fine on my machine.
Please enlighten me, if I need to change anything.

@maralorn

can I do something to get a Haskell label on my PR?

I think ofborg is supposed to do it automatically.  I'm not sure what is happening here.

the build error @GrahamcOfBorg is complaining about

I actually just figured out how to reproduce this locally, and I wrote a short blog post about it:
https://functor.tokyo/blog/2020-06-02-evaluating-nixpkgs
I think you're right that these errors aren't related to this PR.  However, they are things that prevent PRs like https://github.com/NixOS/nixpkgs/pull/90032 from being merged in.  It is mostly @peti who fixes these up, but if you were willing to send a PR fixing some of these evaluation errors, @peti and I would really appreciate it.  (It would make @peti's job easier when he goes and merges haskell-updates into master.)

@peti Upstream has agreed, that the missing dependency on hackage is bad. (https://github.com/reflex-frp/reflex-dom/issues/392#issuecomment-647072593) So I hope we will see a fix there at some point.


Add 'kubefed2 federate' command

kubefed2 federate is intended to create the CRDs and FederatedTypeConfig necessary to federate a given target type.  This should remove the need to commit code to the repo to support types that are not expected to be referred to by controller code.
It will still likely make sense to include sample test fixture and examples in the tree, just not CRDs and FederatedTypeConfigs.
This code has been tested to work, but at present the CRDs are being generated without a validation schema (TODO).  Propagation still works without this.
A first pass of #60.

FYI.. I had to remove the Deployment type to use an older version. When I created Deployment, it did not create the FederatedDeploymentOverride CRD when using the federate command even though it existed before I removed the newer version.

As per a comment in the code, generation of the override CRD is not yet implemented.  This PR is still WIP, so there are gaps yet to be filled.

@danehans The current iteration of this PR enables a simpler federate command:
kubefed2 federate <type/kind/short name>

group/version/namespaced are now determined automatically based on the servers preferred resources (i.e. latest).

group/version/namespaced are now determined automatically based on the servers preferred resources (i.e. latest).

Do I still have the ability to override group and/or version? The reason I ask is b/c I am in the process of federating Istio. As you can see from this manifest, older versions of some resources are being used even-though my k8s cluster is v1.11.

@danehans While it could be possible to choose an older API version, why would that be desirable?
A FederatedTypeConfig defines the version of the API endpoint that will be used in propagating resources to member clusters.  If a resource is provided in a template with an older version (i.e. missing fields found in newer versions as per the Kubernetes convention of backwards compatibility), the default behavior would be provided for those fields regardless of which endpoint was used to create it.

federate now supports the enable and disable subcommands
federate enable supports overrides (via --override-paths)
crd validation is in progress

While it could be possible to choose an older API version, why would that be desirable?

@marun l am in the process of federating Istio v0.8 and 1.x. I want to support both versions because I intend on demonstrating upgrades. The v0.8 manifest that I shared in my previous comment uses older API versions. I could see others doing the same.

I am trying to federate the Istio custom resource stdio and I get the following error:
F1030 17:55:45.006244   67587 federate.go:113] error: Unable to verify that "stdio" is a valid target type: the server could not find the requested resource (get stdioes.config.istio.io)

Here is the api resource:
$ kubectl api-resources | grep std
stdios                                                         config.istio.io                     true         stdio

As you can see, the federate command is looking for the incorrect pluralization of stdio.
Grammer reference:
https://www.grammarly.com/blog/spelling-plurals-with-s-es/
I believe I ran into this same issue with ingress a while back.

Same issue with logentry:
$ kubectl api-resources | grep logentry
logentries                                                     config.istio.io                     true         logentry
$ kubefed2 federate --namespaced=true --group=config.istio.io --version=v1alpha2 --kind=logentry
F1030 18:10:36.478587   69257 federate.go:113] error: Unable to verify that "logentry" is a valid target type: the server could not find the requested resource (get logentrys.config.istio.io)

As per a slack conversation with @danehans, the above reported errors were using a stale version of this PR and do not represent failures with its current state.

@font Thank you for the review, much appreciated!  My apologies for the size of this PR.

@marun apologies on running late on the reviews; When I got to this now it was already merged.
I had few observations/questions which I would point out anyways.

I believe some documentation around this would be warranted soon.
Would it be a good idea to clear it out in help that this should be run against a cluster which has a minimal federation API (say TypeConfig CRD) installed. I mean one can go ahead and run it against a pristine cluster also.
The help line probably does not have that info.
Current context is assumed to be a Kubernetes cluster hosting the federation control plane.
I am fine if there is a plan of documenting it otherwise soon.

Does it make sense to have An optional subtext to arguments description where they are optional

Plan around having a same name Federated CRD installed under different gv? Does it make sense to federate same target type under different gv?

@irfanurrehman

kubefed2 has a dependency on the federation API being installed, not just the added command.  I think you're correct that documentation of this requirement is suggested, and would ask that you add an issue to track it.

All flags are optional (name is an argument, and mandatory), so I'm not sure it makes sense to add gratuitous text to each argument description.

I don't think it makes sense for a given target to have more than a single set of primitives (template, placement, override).  My intention is for all primitives to be created in the primitives.federation.k8s.io group, and only users with a need we haven't anticipated should be changing --primitive-group.  So long as a given deployment uses the same group for all primitives, it won't be possible to define a set of primitives more than once for a given target since CRDs are cluster scoped and the generated name is <target plural name>.<target group>.

Version I'm less clear on, since versioning of CRDs is not something I understand very well.  At this time I would expect the primitives for a given type to be updated with the validation schema of a newer version of the target rather than supplanted by new primitives under a different version.

Created https://github.com/kubernetes-sigs/federation-v2/issues/383 for doc.
Ack.

3. So long as a given deployment uses the same group for all primitives, it won't be possible to define a set of primitives more than once for a given target since CRDs are cluster scoped and the generated name is <target plural name>.<target group>.

Probably I was not completely clear the first time. So currently I can federate say Services under different groups as many number of times as I want. Is this expected? Is different users (or same user) federating the same type under different groups a valid use case? (I haven't come across the use of the same so far).
Lets leave the version discussion for a later date.. :)

There is an added check against federating services multiple times under many different groups for a given federation control plane - the FederatedTypeConfig is also named <target plural name>.<target group>.  Thus, for a given federation, it is only possible to configure propagation/scheduling/(soon status collection) once for a given target type.
It would be possible for user A to generate federated type CRDs under group A, and  userB to generate CRDs under group B, and so long as each of them ran namespaced federation they could have each have FederatedTypeConfig instances for the target types pointing to their desired primitive CRDs.


Azure AD authentication Issuer URL wrong example URL

It says (e.g., "https://login.microsoft.com" for global Azure), also replacing  but according to your support this is the old endpoint and should be https://login.microsoftonline.com. The old URL can make the azure authentication fail in some cases.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 6f1c454d-73bc-5a73-5d3f-1b1ca7625044
Version Independent ID: 552b3f09-6650-e7f7-01f2-a702005b83e9
Content: Configure Azure AD authentication - Azure App Service
Content Source: articles/app-service/configure-authentication-provider-aad.md
Service: app-service
GitHub Login: @cephalin
Microsoft Alias: cephalin

@davidweterings Thanks for the feedback! We are currently investigating and will update you shortly.

@davidweterings Thanks for the feedback! I have assigned the issue to the content author to review further and update the document as appropriate.

The doc has since been updated already. #please-close


Something went wrong. Please try again in a couple of minutes

While using the website, I encountered an error message:
"Something went wrong. Please try again in a couple of minutes or raise an issue on GitHub. Thank you!"
This error prevents me from using the site.
Additional:  This issue occurs every time I visit the site, regardless of refreshing or clearing my cache. I have also tried accessing the site from different devices and browsers, but the error persists.

Your contribution history is private so you wrapped is empty: https://www.githubwrapped.io/Abdullah-Niaz


Drop down menu items

First off, thanks for a great template, documentation and your responsiveness to questions.
I am trying having difficulty creating drop down (?nested?) menu items for the main menu. For instance on my site citizenlabs.org, I would like to have a drop down menu from "About" that would go to our Code of Conduct, Sponsors, and Team.
Find the github repo here.
I have organized these file in the content directory, About.
content
-->about
------>_index.md
------>conduct.md
------>sponsors.md
------>team.md
I have been reviewing and asking questions in the Hugo community and also searching for Hugo templates as examples. Surprisingly I don't see any of the existing templates with drop down navigation menus. I have spent so many hours trying to figure this out, any help would be appreciated.
Thanks

Hi Allen616 I went on your website and it seems that you were able to solve it. Can you tell me how you did it? Thank you

@valkiii I think this is the place you're looking for, right @Allen616 ? https://github.com/citizenlabsgr/homepage/blob/7b210347902b9dddba74748851297f8fa4cf4e9c/config.toml#L74


[Discuss] extend tensorflow backend's switch to match theano backend

It already exists. It's called K.switch().

@farizrahman4u  correct me if I'm wrong, by reading the docstring, K.swtich need a condition if a scalar, not applies element-wise selection.

Just look at the code for K.switch. It does the same thing as you did in this PR. It simply maps to T.switch and tf.select.

For the theano backend, yes.
But for tensorflow, it's not the case. The tf backend invokes cond which need condition to be scalar.

Then fix K.switch's tensorflow backend.

My two cents: this seems like a bad idea, because it will break code (such as mine) that depends on K.switch being an elementwise operation, which it currently is on Theano.
I agree more with farizrahman4u's suggestion: since switch is the name of an existing Theano function and the name doesn't exist in TensorFlow, wouldn't it make more sense to make K.switch in TensorFlow similar to the Theano function? You're now inventing a definition of K.switch that doesn't match either backend.
You could add a K.ifelse or something similar to handle the scalar-condition case. And if you really want optimize K.switch, I'd suggest making it check whether the input is a scalar and use switch or ifelse depending on that (but this seems like an optimization that should be included in Theano).

@gvtulder that's a good point.

@fchollet @gvtulder yes, extending K.switch is much better. But there is still one thing:
theano's switch is basically more powerful than tensorflow's select.
For theano's switch, condition can be a scalar or tensor of the same rank as if/then expression's.
For tensorflow's seleect, condition can be a scalar or vector.
So should we leave it that way (as the new PR).
or should we try to make tf backend does as much as theano backend, which seems nontrivial (th swtich support broadcast of arbitrary dimension).

@DingKe Here is the best thing to do IMO:

Leave Theano backend for K.switch as is
Fix TF backend for K.switch so that its behavior matches with Theano backend
Optional : In theano backend of K.switch, if condition is scalar, use ifelse instead of T.switch

@DingKe: That looks better. I don't really understand why you think Theano's switch is much more powerful than tf.select, but I'm not a TensorFlow user. The TensorFlow documentation seems to say that tf.select also accept tensors the same size as the output, so it should be possible to do an elementwise operation with that.

I don't think it is a good idea to convert by default switch to ifelse for
Theano. Currently, there is some extra overhead as we don't have c code for
it. As we don't have a c interface for "lazy op", it is not trivial to add
it, not super difficult, just not trivial and it isn't a current priority.
On Mon, Dec 5, 2016 at 11:50 AM, Gijs van Tulder<EMAIL_ADDRESS>wrote:

@DingKe https://github.com/DingKe: That looks better. I don't really
understand why you think Theano's switch is much more powerful than
tf.select, but I'm not a TensorFlow user. The TensorFlow documentation
seems to say that tf.select also accept tensors the same size as the
output, so it should be possible to do an elementwise operation with that.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/fchollet/keras/pull/4519#issuecomment-264823386, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AALC-zczr3Lwgs6KwCSzX0SM4l1JFgCmks5rE-yMgaJpZM4K9X4C
.

TF select is getting deprecated and replaced with where. So that's what we should be normalizing on.

Basically, tf's switch is aligned to theano's.
When condition is scalar, it's a if/else condition (the old interface);
Otherwise condition should be the same shape with then/else expression, and a element-wise selection is preformed.
@fchollet
(@farizrahman4u: not use ifelse for theano according to @nouiz.)

It looks like the Travis failure is because tf.where works differently in TensorFlow v0.11. Perhaps you can make a fallback option that uses tf.select if you get this TypeError?

Closing outdated PR. If you still care about the content of the PR, please submit a new PR to master, updated for the Keras 2.0 API.


Remove warnings from bench/scripts/micro/to_sql_bench.exs

:heart: :green_heart: :blue_heart: :yellow_heart: :purple_heart:


metric of cpu usagePercent is still 0.

Please answer these questions before submitting your issue.

Why do you submit this issue?
[ ] Question or discussion
[x] Bug
[ ] Requirement
[ ] Feature or performance improvement

Bug

Which version of SkyWalking, OS and JRE?
version:6.0-GA;
os:windows
jre:1.8

Requirement or improvement

Please describe about your requirements or improvement suggestions.

We have the metrics of CPU in SkyWalking 6.1

We have the metrics of CPU in SkyWalking 6.1

Look at source code,v6.1 is ok,it has been repaired.


infer literal in discriminatedUnion fails typecheck but not parse

I want to typecheck the object before parsing it to validate it.
But the type generated from a discriminatedUnion does not seem to match the zod parse() result.
const myUnion = z.discriminatedUnion("status", [
  z.object({ status: z.literal("success"), data: z.string() }),
  z.object({ status: z.literal("failed"), error: z.instanceof(Error) }),
]);

const data = { status: "success", data: "yippie ki yay" };

type TypeFromSchema<T> = T extends ZodType<infer U> ? U : never;

function test<T extends ZodType>(data: TypeFromSchema<T>, schema: T) {
  return myUnion.safeParse(data); // works
}

const result = test(data, myUnion);  // typecheck fails

Result:
Argument of type '{ status: string; data: string; }' is not assignable to parameter of type '{ status?: "success"; data?: string; } | { status?: "failed"; error?: Error; }'.
  Type '{ status: string; data: string; }' is not assignable to type '{ status?: "success"; data?: string; }'.
    Types of property 'status' are incompatible.
      Type 'string' is not assignable to type '"success"'.

Expected result:
I'm probably wrong here but I would expect no type errors. Having the type of status be a literal too and not a string.
codesandbox.io

You'll need to declare data as const for that to succeed.
const data = { status: "success", data: "yippie ki yay" };


enable race detection in travis build

Currently, there seems to not be any race condition checking while building (go test -race).
It would be useful to include race detection check in the travis build.
We should be able to enable this by specifying gobuild_args: -race in travis.yml file.

Seems reasonable to me. I would accept a PR activating that, assuming it passes. Thank you.