Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
343 changes: 239 additions & 104 deletions site/docs/concepts/information-supply-chain.md

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -363,7 +363,7 @@ Tom discovers that although the refrigeration units were catalogued in the vario

Tom looks at the existing [open metadata types](/types) and cannot find a good place to store details of their refrigeration units. She proposes to add a new RefrigerationUnit type to Egeria. It would inherit from [Infrastructure](/types/0/0010-Base-Model) and include a new attribute called *coolantType*.

![New Type Definition](refrigeration-unit-type.svg)
![New Type Definition (as proposed by Tom)](refrigeration-unit-type-proposed.svg)

This type would be used to store details of the refrigeration units. She takes her idea to Erin, who calls a meeting to discuss it. There are two items on the agenda:

Expand All @@ -376,23 +376,37 @@ This type would be used to store details of the refrigeration units. She takes
The decision to use Egeria for this data was easy. Egeria supports the ongoing integration of the company's asset management systems, and providing a common language for describing assets across the company. It then was well placed to support the governance of this equipment.

The proposed new type generated more discussion. Erin had picked up on Tom's broader plan to bring details of other equipment relevant to sustainability. She suggested that they should define a new type called Equipment, and then have RefrigerationUnit inherit from Equipment.

![New Type Definitions (as agreed)](refrigeration-unit-type.svg)

This means that they can have common management of all Equipment, beyond just refrigeration units.

The others agreed and [Lemmie Stage](/practices/coco-pharmaceuticals/personas/lemmie-stage) was asked to add the new type to Egeria.

#### Adding a new open metadata type for refrigeration units

There are three steps to adding a new open metadata type:

* Create a new type definition in an [Open Metadata Archive](/concepts/open-metadata-archive).
* Add the new archive to the start-up list of archive files to load at [Metadata Access Store](/concepts/metadata-access-store) start-up.
* Restart the metadata access store to load the new archive.
There are two approaches to adding a new open metadata type. It can be added via an [Open Metadata Archive](/guides/developer/open-metadata-archives/defining-new-types/) or via the [Metadata Expert API](/services/omvs/metadata-expert/overview). Lemmie decides to use the API. The [Going CFC-Free Coco Workbook](https://github.com/odpi/egeria-workspaces/tree/main/coco-workbooks/3.%20sustainability/going-cfc-free) shows the code to create these new types and load the refrigeration unit data into Egeria.

![Lemmie defining the new RefrigerationUnit type](lemmie-defining-refrigeration-unit.png)

???+ info "Viewing the new types in Egeria"
You can view the new types in Egeria's type explorer which is part of [Egeria Explorer](/user-interfaces/egeria-explorer/overview).

This is the Equipment type.
![Lemmie's new Equipment type](equipment-typedef.png)

This is the refrigeration unit type.
![Lemmie's new RefrigerationUnit type](refrigeration-unit-typedef.png)

Because Equipement inherits from Infrastructure, it is visible through [The Catalog](/user-interfaces/the-catalog/overview). Notice the new *Equipment* tab and the refrigeration units below.
![Refrigeration units in The Catalog](refrigeration-unit.png)

The new properties are returned directly in the JSON when retrieving them through the [Asset Maker API](/services/omvs/asset-maker/overview). New properties are added using [extendedProperties](/parameters/overview/#typename-and-extendedproperties).
![Refrigeration Unit JSON](refrigeration-unit-json.png)

### Awarding the First CFC-Free certificate

Once the RefrigerationUnit was installed in Egeria, Tom was able to load the refrigeration unit data that she had collected from the different sites. She showed the results to her boos, [Reggie Mint](/practices/coco-pharmaceuticals/personas/reggie-mint) to request budget to replace the CFC-containing units with CFC-free alternatives. He agrees because it matches their strategy and the budget is approved.
Once the RefrigerationUnit was installed in Egeria, Tom was able to load the refrigeration unit data that she had collected from the different sites. She showed the results to her boss, [Reggie Mint](/practices/coco-pharmaceuticals/personas/reggie-mint) to request budget to replace the CFC-containing units with CFC-free alternatives. He agrees because it matches their strategy and the budget is approved.

Then the race is on to acquire the CFC-free alternatives. Which location will be the first to be CFC-Free? Not surprisingly, the New York lab is the first, because [Zach Now](/practices/coco-pharmaceuticals/personas), forewarned of the initiative, had gone ahead and ordered new units before Tom had the data into Egeria! The other locations followed over the next few months allowing Tom to declare the company CFC-Free.

Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -1719,7 +1719,7 @@
</mxGraphModel>
</diagram>
<diagram id="eGc2Fq3J0zg0qtuEqHZz" name="refrigeration-types">
<mxGraphModel dx="1222" dy="832" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
<mxGraphModel dx="1211" dy="817" grid="1" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="827" pageHeight="1169" math="0" shadow="0">
<root>
<mxCell id="0" />
<mxCell id="1" parent="0" />
Expand Down Expand Up @@ -1761,8 +1761,8 @@
<mxCell id="c8bnrQ5nSINEYd2gY4Qw-4" parent="1" style="html=1;strokeColor=#d6b656;align=center;fillColor=#fff2cc;gradientColor=#ffd966;sketch=1;curveFitting=1;jiggle=2;" value="«entity»&lt;br&gt;&lt;b&gt;Equipment&lt;/b&gt;" vertex="1">
<mxGeometry height="50" width="240" x="440" y="800" as="geometry" />
</mxCell>
<mxCell id="c8bnrQ5nSINEYd2gY4Qw-5" parent="1" style="rounded=0;whiteSpace=wrap;html=1;strokeColor=#000000;gradientColor=#ffffff;align=left;sketch=1;curveFitting=1;jiggle=2;" value="lastInspectionDate : date" vertex="1">
<mxGeometry height="40" width="239" x="440" y="850" as="geometry" />
<mxCell id="c8bnrQ5nSINEYd2gY4Qw-5" parent="1" style="rounded=0;whiteSpace=wrap;html=1;strokeColor=#000000;gradientColor=#ffffff;align=left;sketch=1;curveFitting=1;jiggle=2;" value="make : string&lt;div&gt;model : string&lt;/div&gt;&lt;div&gt;purchaseDate : string&lt;/div&gt;&lt;div&gt;&lt;span style=&quot;background-color: transparent; color: light-dark(rgb(0, 0, 0), rgb(255, 255, 255));&quot;&gt;usedFor : string&lt;/span&gt;&lt;/div&gt;" vertex="1">
<mxGeometry height="90" width="239" x="440" y="850" as="geometry" />
</mxCell>
<mxCell id="c8bnrQ5nSINEYd2gY4Qw-6" parent="1" style="html=1;strokeColor=#d6b656;align=center;fillColor=#fff2cc;gradientColor=#ffd966;sketch=1;curveFitting=1;jiggle=2;" value="«entity»&lt;br&gt;&lt;b&gt;Asset&lt;/b&gt;" vertex="1">
<mxGeometry height="50" width="240" x="440" y="590" as="geometry" />
Expand All @@ -1773,10 +1773,16 @@
</mxGeometry>
</mxCell>
<mxCell id="c8bnrQ5nSINEYd2gY4Qw-8" parent="1" style="html=1;strokeColor=#d6b656;align=center;fillColor=#fff2cc;gradientColor=#ffd966;sketch=1;curveFitting=1;jiggle=2;" value="«entity»&lt;br&gt;&lt;b&gt;RefrigerationUnit&lt;/b&gt;" vertex="1">
<mxGeometry height="50" width="240" x="440" y="950" as="geometry" />
<mxGeometry height="50" width="240" x="440" y="1000" as="geometry" />
</mxCell>
<mxCell id="c8bnrQ5nSINEYd2gY4Qw-9" parent="1" style="rounded=0;whiteSpace=wrap;html=1;strokeColor=#000000;gradientColor=#ffffff;align=left;sketch=1;curveFitting=1;jiggle=2;" value="refrigerantType : string&lt;div&gt;refrigerantVolume : int&lt;/div&gt;&lt;div&gt;refrigerantPercentageLossPerYear : string&lt;/div&gt;" vertex="1">
<mxGeometry height="80" width="239" x="440" y="1050" as="geometry" />
</mxCell>
<mxCell id="R_xSUlrNh277-_QSq5B--1" parent="1" style="text;html=1;whiteSpace=wrap;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;fontStyle=1" value="Proposed" vertex="1">
<mxGeometry height="30" width="60" x="169.5" y="520" as="geometry" />
</mxCell>
<mxCell id="c8bnrQ5nSINEYd2gY4Qw-9" parent="1" style="rounded=0;whiteSpace=wrap;html=1;strokeColor=#000000;gradientColor=#ffffff;align=left;sketch=1;curveFitting=1;jiggle=2;" value="refrigerantType : string&lt;div&gt;fullChargeCapacity : int&lt;/div&gt;&lt;div&gt;fullChargeCapacityUnits : string&lt;br&gt;&lt;div&gt;leakRatePercentage : int&lt;/div&gt;&lt;/div&gt;" vertex="1">
<mxGeometry height="110" width="239" x="440" y="1000" as="geometry" />
<mxCell id="R_xSUlrNh277-_QSq5B--2" parent="1" style="text;html=1;whiteSpace=wrap;strokeColor=none;fillColor=none;align=center;verticalAlign=middle;rounded=0;fontStyle=1" value="Agreed" vertex="1">
<mxGeometry height="30" width="60" x="529.5" y="520" as="geometry" />
</mxCell>
</root>
</mxGraphModel>
Expand Down
10 changes: 3 additions & 7 deletions site/snippets/parameters/extended-properties-example.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
<!-- Copyright Contributors to the Egeria project. -->


For example, the [Data Manager OMAS](/services/omas/data-manager/overview) has support for maintaining metadata about *Topics*. Topics are special types of [*assets*](/concepts/asset) that provide a destination for exchanging events.
For example, the [Asset Maker API](/services/omvs/asset-maker/overview) has support for maintaining metadata about *Topics*. Topics are special types of [*assets*](/concepts/asset) that provide a destination for exchanging events.

Model [0223 Events and Logs](/types/2/0223-Events-and-Logs) extends the `Topic` type with a new type called `KafkaTopic` that introduces two new properties: `partitions` and `replicas`. `KafkaTopic` is used to represent a topic that is owned and managed by [*Apache Kafka*](https://kafka.apache.org/). The full type hierarchy for `KafkaTopic` is shown in Figure 1.

Expand Down Expand Up @@ -47,14 +47,10 @@ Finally, any additional properties that the organization wants to store that are
additionalProperties.put("jsonPayload", "true");
topicProperties.setAdditionalProperties(additionalProperties);

String topicGUID = dataManagerClient.createTopic(userId,
eventBrokerGUID
eventBrokerName,
eventBrokerIsHome,
topicProperties);
String topicGUID = assetMakerClient.createAsset(..., topicProperties, ...);

```

!!! info "Further information"
* [Data Manager OMAS](/services/omas/data-manager/overview)
* [Asset Maker API](/services/omvs/asset-maker/overview)

13 changes: 13 additions & 0 deletions site/snippets/release-notes/6-3.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,17 @@ _**These are DRAFT release notes. The notes will be updated until and when 6.3
??? functional "Open Lineage Cataloguer: table-level lineage is a DataMapping"
The [Open Lineage Cataloguer](/features/lineage-management/overview/#egerias-open-lineage-support) now uses the relationship type that matches the elements it links. Lineage between assets - two data sets, or a data set and a process - is a *DataFlow*. Lineage that has a schema attribute such as a relational table at either end - between two tables, or between a table and the process that reads or writes it - is a *DataMapping*, and a *DataFlow* is added between the table's asset (its schema or database) and the other end so that the lineage is visible at the asset level too. Previously the cataloguer created *DataFlow* relationships between tables and processes, and both a *DataFlow* and a *DataMapping* between two tables.

??? functional "Information supply chain graphs split by layer"
An [information supply chain](/concepts/information-supply-chain) retrieved through the [Solution Architect API](/services/omvs/solution-architect/overview) with its implementation now comes with five mermaid graphs instead of one, because a single graph of a large supply chain was often too big for mermaid to display:

* *iscImplementationMermaidGraph* - the supply chain's context, the design of its solution components and the status of their implementation. The Context, Design and Status areas are now drawn left to right.
* *iscDataMeshMermaidGraph* - the [digital products](/concepts/digital-product) and the dependencies between them.
* *iscSystemFabricMermaidGraph* - the IT infrastructure and software capabilities that the supply chain runs on.
* *iscDataFabricMermaidGraph* - the data and processes linked by data lineage (*DataLineageRelationship* and its subtypes), *ControlFlow* and *ActionRequester* relationships.
* *iscDataMappingMermaidGraph* - the elements linked by *DataMapping* relationships, such as the columns of tables.

Each [layer of lineage](/features/lineage-management/overview/#the-layers-of-lineage) is drawn inside a subgraph. The solution components linked to its elements, through *ImplementedBy* for example, are drawn outside the subgraph with their links, and so are the digital products linked to the data fabric. An element's anchor, if it is part of the supply chain, is linked to it with a line labelled *Anchor for*: every anchor in the data mapping graph, solution components in each of the other layers, and digital products in the data mesh. The sample information supply chain in the [Core Content Pack](/content-packs/core-content-pack/overview), `SAMPLE Information Supply Chain - Deployment Status Demonstration`, now shows every graph.

??? functional "New test suite: anchor-fvt"
A new functional verification test suite, *anchor-fvt*, exercises [anchor management](/features/anchor-management/overview) against both the PostgreSQL and the in-memory repositories: the scope of a delete, archive and purge, both of an anchor and of an anchored member, templated creates, restore and undo. Its fixture is a catalogued database schema with its tables, columns, foreign keys, lineage, connection and a linked glossary term.

Expand Down Expand Up @@ -132,12 +143,14 @@ _**These are DRAFT release notes. The notes will be updated until and when 6.3
* Darwin no longer records an exception for a *DigitalProductDependency* on or from a digital product with no assets among its members, because no lineage could prove or disprove it; it is counted as unchecked instead.
* The parameters of audit log messages `DARWIN-PRODUCT-DEPENDENCY-MANAGER-0003`, `-0010` and `-0012` to `-0015` have changed: they name the relationship type, and the refresh tallies include the unproven and unchecked relationships. Anything that parses these messages needs updating.
* The instance header has a new field, *lastOperationId*, which is also returned in *ElementVersions* and can be searched on as a header property. The PostgreSQL repository adds a `last_operation_id` column to its entity, relationship and classification tables automatically when the server starts. Instances written before the upgrade have no value, so a delete made before the upgrade is restored one element at a time.
* The *iscImplementationMermaidGraph* of an information supply chain no longer includes the Data Mesh, Data Fabric and System Fabric areas; they have moved to graphs of their own - see *Information supply chain graphs split by layer* above.
* Responses no longer include numeric and boolean properties that are not set; earlier releases returned 0 or false for them. Callers that need a value should use the property's default value - see *Numbers and flags that are not set stay unset* above. The defaults are the values the beans used to start with, apart from *allowsDuplicateValues*, which defaults to true: a *SchemaAttribute* with no value was stored as false by earlier releases, and is now read as true.

??? danger "API changes"
* *claimEngineAction*, *getActiveClaimedEngineActions*, *updateEngineActionStatus*, *updateActionTargetStatus* and *recordCompletionStatus* have moved from the [Automated Curation API](/services/omvs/automated-curation/overview) to the new [Governance Engine API](/services/omvs/governance-engine/overview). Their paths, request bodies and responses are unchanged, so callers, including pyegeria, replace the `automated-curation` URL marker with `governance-engine`. For example, `.../api/open-metadata/automated-curation/engine-actions/{engineActionGUID}/claim` becomes `.../api/open-metadata/governance-engine/engine-actions/{engineActionGUID}/claim`.
* *ActionControlInterface* has a new method, *cancelGovernanceActionProcess*. Other implementations of the interface need to add it.
* *MetadataElementInterface* and *OpenMetadataClient* have four new methods: *undoMetadataElementUpdate*, *undoRelationshipUpdate*, *restoreMetadataElementInStore* and *restoreRelationshipInStore*. Other implementations of the interface need to add them.
* The information supply chain's implementation graph is now returned in JSON as `iscImplementationMermaidGraph`. Earlier releases returned it as `iscimplementationMermaidGraph`, so callers that read that name need to change. The new graphs are `iscDataMeshMermaidGraph`, `iscSystemFabricMermaidGraph`, `iscDataFabricMermaidGraph` and `iscDataMappingMermaidGraph`.
* The *RetentionProperties* JSON type name is now `RetentionProperties`; requests that used `RetentionProperties.class` need to change.
* *OpenLineageDataSetResolver.MatchedElement* in the Open Integration Framework has a new component, *matchedBy*, saying how the element was found to describe the Open Lineage dataset. Code that constructs it needs to supply the new value.
* The numeric and boolean properties of the Open Metadata Framework property beans are now boxed types (`Integer`, `Long`, `Float`, `Boolean`), and may return null. Code that uses them as primitives - in arithmetic, a condition, or an `int` variable - throws a *NullPointerException* when the value is not set; use the property's default instead, for example `OpenMetadataProperty.CONFIDENCE.intValueOrDefault(properties.getConfidence())`. A setter that takes a `Long` no longer accepts an `int`, so a literal needs an `L`, such as `setKarmaPoints(10L)`. The `remove...` methods of *OpenMetadataPropertyConverterBase* for these properties also return boxed types.
Expand Down
Loading