Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions .github/workflows/goat-obs-checks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,6 +56,13 @@ on:
- 'ops/relay/health/test_relay_metrics_ingest_allowlist.py'
- 'ops/relay/health/test_relay_metrics_ingest.py'
- 'ops/relay/health/test_atomic_ingest.py'
# The peer-names writer carries each peer's owner (peer-identity-resolution.md).
- 'ops/relay/health/relay-peer-names.py'
- 'ops/relay/health/test_relay_peer_names_owner.py'
- 'ops/relay/health/test_peer_names_role.py'
- 'ops/ansible/roles/relay-health/tasks/peer-names.yml'
- 'ops/identity/peerowner.py'
- 'ops/identity/test_peerowner.py'
- 'ops/ci/check-grafana-dashboard-promql.sh'
- 'ops/ci/check-textfile-exposition.sh'
- 'ops/ci/check-blackbox-covers-registry.sh'
Expand Down Expand Up @@ -277,3 +284,10 @@ jobs:
run: |
python -m py_compile relay-metrics-ingest.py
python -W error::ResourceWarning -m unittest test_atomic_ingest -v

- name: peer owners (shared client + relay-peer-names writer)
run: |
python -m pip install --quiet jinja2 pyyaml
python -m py_compile ops/relay/health/relay-peer-names.py
python -W error::ResourceWarning ops/identity/test_peerowner.py
cd ops/relay/health && python -W error::ResourceWarning -m unittest test_relay_peer_names_owner test_peer_names_role -v
67 changes: 50 additions & 17 deletions docs/design/peer-identity-resolution.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,10 +73,10 @@ here, so that every consumer gets them.
## The owner comes from the record, not the roster

**Status:** the resolver's record tiers for machines and for slot-linked
devices are built (§"The resolver"). The identity handle and nationality
stamps, declared owners for infrastructure, the ownership audit, the host
carrier and the surface states are designed, not built; each names where it
will be built.
devices are built (§"The resolver"), and so is the host carrier (§"The
carrier"). The identity handle and nationality stamps, declared owners for
infrastructure, the ownership audit and the surface states are designed, not
built; each names where it will be built.

**The rule.** No machine, and above all no router, holds mesh access without a
recorded responsible person and organization. A surface that cannot show the
Expand Down Expand Up @@ -220,19 +220,52 @@ infrastructure, leave the person devices with no identity and the peers whose
name carries no owner. Each of those needs an owner recorded or its access
removed. That is an operator action, not code.

### The carrier to the host surfaces (designed, not built)

The relay-health `relay-peer-names` writer already calls the batch resolver
every 5 minutes on the management host and writes `peer-names.json`. Two
changes:

- **It writes the `owner` block per peer.** When the portal refuses or cannot
be reached, it writes `owner.source: "unread"` with the HTTP status. Today it
writes the file with no owner fields and no marker, so a refused key is
invisible.
- **A deploy check fails when the portal refuses the writer's key.** The
writer and peerview send one key on at least one goatnet, and the existing
`assert-portal-key-accepted.yml` is included by neither role.
### The carrier to the host surfaces (built)

The relay-health `relay-peer-names` writer runs every 5 minutes on the
management host and writes `peer-names.json`. A failed owner read never
becomes "no owner".

- **It asks about every peer,** machines and infrastructure included. Names go
in chunks of 500, under the portal's 1 MiB request cap.
- **It writes each card's `owner` block.** The block carries no e-mail or
display name; `ory_id` is kept as before. The shared client
`ops/identity/peerowner.py` carries the block on every row and never raises:
a body that is not a peers map is `unread` for every name.
- **The portal says when it could not read its own sources.** The batch
answer carries `owner_sources: {ok, failed}`. When a layer the owner tiers
read did not answer (the registry, the slot aliases, the enrolled machines
or the identity layer), `ok` is false and every row's owner is `unread`,
naming the layer.
- **Other answers that are unread, not "none":**
- the portal cannot be asked (no key on the host, refused, unreachable);
- its sources failed;
- it is older than the `owner` block;
- it sends an owner block without a `source`.

When every card is unread, the file says so once:
`owner_read: {state: "unread", reason, at}`.
- **The cloud relay-health box gets the owner state only.** It receives the
names it always had plus each card's owner state (`source`, and whether an
owner exception covers it), and the file-level `owner_read` and
`generated_at`. These ride in a reserved key that an older ingest stores
harmlessly and a current one lifts to the top of its file. No person,
identity id, organization or nationality leaves the management host.
- **An over-cap or refused cloud body is a failure you can see.** A body over
the ingest's cap (262144 bytes) is not sent, and the writer exits non-zero
after writing the on-host file. A cloud ingest that refuses the body does
the same. A cloud box that cannot be reached is logged only.
- **The deploy proves the key on the route the writer uses.** The relay-health
role sends `POST /api/access/peers` with an empty list and the writer's key.
- A refused key (401 or 403) fails the deploy.
- A portal that is down or not found is a warning, and the health dashboard
still deploys.
- The writer enabled with no key fails the deploy, unless
`relay_peer_names_owner_enabled: false` switches owners off on purpose.

A host writer's key can drift from the portal's root key, and the portal's
handle-map guard never compares the root key, so this check is the only
thing that catches it.

No new credential: the writer already holds a key.

Expand Down
5 changes: 5 additions & 0 deletions ops/ansible/roles/relay-health/defaults/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -280,6 +280,11 @@ relay_peer_names_id_console_url: "" # id-admin-console base, e.g. https://admi
relay_peer_names_portal_container: "" # when portal_url unset: docker-inspect this container's IP on the net below (e.g. goat-zenoh-portal)
relay_peer_names_portal_net: "mgmt" # docker network the portal app listens on
relay_peer_names_portal_port: "8090" # portal app port (not the oauth2/ADMINS gate)
# The writer reads each peer's owner from the portal (ADR 1249). On, a
# deploy with no relay_peer_names_portal_key fails rather than ship a writer
# whose owners are "not read" forever; false deploys the writer without owners
# on purpose.
relay_peer_names_owner_enabled: true

# --- Host-net health ingest (site-uplink diagnosis) --------------
# Comma-separated site network hosts (mgmt-hosts/Thors) allowed to POST
Expand Down
90 changes: 90 additions & 0 deletions ops/ansible/roles/relay-health/tasks/peer-names.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,96 @@
- { src: "{{ relay_health_src_dir }}/relay-peer-names.py", dest: relay-peer-names.py, mode: "0755" }
- { src: "{{ playbook_dir }}/../identity/peerowner.py", dest: peerowner.py, mode: "0644" }

# The writer puts each peer's owner on every card (peer-identity-resolution.md
# §"The carrier"). A key the portal refuses makes every card `owner: unread`
# while the timer still succeeds, so the deploy asks the portal, with the
# key the writer presents and on the route the writer uses (the operator-only
# `POST /api/access/peers`), before reporting success. Only a REFUSED key
# fails the deploy: a portal that is down or not found is said loudly and
# does not block the health dashboard, whose job does not depend on it.
# A host writer's key can drift from the portal's root key without anything
# else noticing, because the portal's handle-map guard never compares it.
- name: relay-health | peer-names — the writer needs a portal key to read owners (ADR 1249)
ansible.builtin.fail:
msg: >-
relay-peer-names is enabled and relay_peer_names_portal_key is empty, so
every peer's owner would be "not read" on every surface, permanently.
Set the key, or set relay_peer_names_owner_enabled: false to deploy the
writer without owners on purpose.
when:
- relay_peer_names_owner_enabled | default(true) | bool
- relay_peer_names_portal_key | default('') | length == 0

- name: relay-health | peer-names — find the portal app address
ansible.builtin.command:
argv:
- docker
- inspect
- -f
- "{{ '{{' }}(index .NetworkSettings.Networks \"{{ relay_peer_names_portal_net }}\").IPAddress{{ '}}' }}"
- "{{ relay_peer_names_portal_container }}"
register: relay_peer_names_portal_ip
changed_when: false
failed_when: false
when:
- relay_peer_names_portal_key | default('') | length > 0
- relay_peer_names_portal_url | default('') | length == 0
- relay_peer_names_portal_container | default('') | length > 0

- name: relay-health | peer-names — the portal address the writer will use
ansible.builtin.set_fact:
relay_peer_names_probe_url: >-
{{ relay_peer_names_portal_url if (relay_peer_names_portal_url | default('') | length > 0)
else ('http://' ~ (relay_peer_names_portal_ip.stdout | default('') | trim) ~ ':' ~ relay_peer_names_portal_port
if (relay_peer_names_portal_ip.stdout | default('') | trim | length > 0) else '') }}
when: relay_peer_names_portal_key | default('') | length > 0

- name: relay-health | peer-names — ask the owner route with the writer's key
ansible.builtin.uri:
url: "{{ relay_peer_names_probe_url | trim }}/api/access/peers"
method: POST
body_format: json
body: {"netbird_names": []}
headers:
Authorization: "Bearer {{ relay_peer_names_portal_key }}"
status_code: [200, 401, 403]
timeout: 15
validate_certs: false
register: relay_peer_names_probe
until: relay_peer_names_probe.status | default(-1) in [200, 401, 403]
retries: 6
delay: 5
changed_when: false
failed_when: false
no_log: true
when:
- relay_peer_names_portal_key | default('') | length > 0
- relay_peer_names_probe_url | default('') | trim | length > 0

- name: relay-health | peer-names — fail when the portal refuses the writer's key
ansible.builtin.fail:
msg: >-
The portal at {{ relay_peer_names_probe_url | trim }} refused the key
relay-peer-names presents (HTTP {{ relay_peer_names_probe.status }} on
POST /api/access/peers, an operator route). Every peer's owner would be
"not read". Re-render relay_peer_names_portal_key from the portal's
current operator key, then re-run.
when:
- relay_peer_names_probe is defined
- relay_peer_names_probe.status | default(-1) in [401, 403]

- name: relay-health | peer-names — say so when the portal cannot be reached
ansible.builtin.debug:
msg: >-
WARNING: relay-peer-names could not reach the portal
({{ (relay_peer_names_probe_url | default('') | trim) or 'no address: the portal container was not found' }}).
The health dashboard deploys; every peer's owner reads "not read" until
the portal answers. The writer retries every cycle.
when:
- relay_peer_names_portal_key | default('') | length > 0
- (relay_peer_names_probe_url | default('') | trim | length == 0)
or (relay_peer_names_probe.status | default(-1) not in [200, 401, 403])

- name: relay-health | peer-names resolver .env
ansible.builtin.copy:
dest: "{{ relay_health_dir }}/relay-peer-names.env"
Expand Down
56 changes: 51 additions & 5 deletions ops/goat-zenoh-portal/internal/api/access.go
Original file line number Diff line number Diff line change
Expand Up @@ -315,8 +315,10 @@ func (s *Server) handleAccessPeer(w http.ResponseWriter, r *http.Request) {
return
}
src, warn := s.collectAccessSources(r.Context())
resp := resolvePeerName(src, name)
out := peersAnswer(src, warn, []string{name})
resp := out["peers"].(map[string]map[string]any)[name]
resp["notes"] = access.NoteTexts(warn)
resp["owner_sources"] = out["owner_sources"]
writeJSON(w, http.StatusOK, resp)
}

Expand All @@ -333,15 +335,59 @@ func (s *Server) handleAccessPeersBatch(w http.ResponseWriter, r *http.Request)
return
}
src, warn := s.collectAccessSources(r.Context())
out := make(map[string]map[string]any, len(body.NetbirdNames))
for _, name := range body.NetbirdNames {
writeJSON(w, http.StatusOK, peersAnswer(src, warn, body.NetbirdNames))
}

// ownerSourceKinds are the layers the owner tiers read: the registry (vendor
// rows), the slot aliases with their recorded identities, the enrolled
// machines and the identity layer. When one of them did not answer, a missing
// owner is not evidence of no owner.
var ownerSourceKinds = map[string]bool{
"registry_unavailable": true,
"slot_aliases_unavailable": true,
"machines_unavailable": true,
"identity_layer_unavailable": true,
"identity_layer_error": true,
}

// ownerSourcesFailed lists the owner-tier layers that did not answer.
func ownerSourcesFailed(warn []access.Note) []string {
var failed []string
for _, n := range warn {
if ownerSourceKinds[n.Kind] {
failed = append(failed, n.Kind)
}
}
return failed
}

// peersAnswer resolves every name and states, machine-readably, whether the
// owner sources answered (`owner_sources: {ok, failed}`). When any did not,
// every row's owner is `unread` with the reason: a failed read must never
// become "no owner", a page, or an owner exception (ADR 1249). The rest of the
// row stays as resolved, for consumers that read only the person fields.
func peersAnswer(src access.Sources, warn []access.Note, names []string) map[string]any {
failed := ownerSourcesFailed(warn)
out := make(map[string]map[string]any, len(names))
for _, name := range names {
name = strings.TrimSpace(name)
if name == "" {
continue
}
out[name] = resolvePeerName(src, name)
row := resolvePeerName(src, name)
if len(failed) > 0 {
row["owner"] = map[string]any{
"source": "unread",
"reason": "the portal could not read " + strings.Join(failed, ", ") + "; the owner is unknown, not absent",
}
}
out[name] = row
}
return map[string]any{
"peers": out,
"notes": access.NoteTexts(warn),
"owner_sources": map[string]any{"ok": len(failed) == 0, "failed": failed},
}
writeJSON(w, http.StatusOK, map[string]any{"peers": out, "notes": access.NoteTexts(warn)})
}

// resolvePeerName does one peer→person resolution against an assembled source
Expand Down
44 changes: 44 additions & 0 deletions ops/goat-zenoh-portal/internal/api/access_owner_sources_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package api

import (
"strings"
"testing"

"github.com/usareur-af/goatcore/ops/goat-zenoh-portal/internal/access"
)

// Review of #977, item 1: a layer the owner tiers read that did not answer
// arrived as HTTP 200 with every owner "none". The batch answer now says so
// machine-readably, and every row's owner is unread.
func TestPeersAnswerSaysWhenAnOwnerSourceFailed(t *testing.T) {
src := recordSources()
ok := peersAnswer(src, nil, []string{"machine-anvil-pod-7"})
if s := ok["owner_sources"].(map[string]any); s["ok"] != true {
t.Fatalf("owner_sources = %v, want ok", s)
}
row := ok["peers"].(map[string]map[string]any)["machine-anvil-pod-7"]
if row["owner"].(map[string]any)["source"] != "record" {
t.Fatalf("owner = %v, want record when every source answered", row["owner"])
}

for _, kind := range []string{"registry_unavailable", "slot_aliases_unavailable", "machines_unavailable", "identity_layer_error", "identity_layer_unavailable"} {
warn := []access.Note{{Kind: kind, Severity: access.SeverityError, Text: kind}}
out := peersAnswer(src, warn, []string{"machine-anvil-pod-7", "edge-gw-unlisted"})
s := out["owner_sources"].(map[string]any)
if s["ok"] != false || strings.Join(s["failed"].([]string), ",") != kind {
t.Fatalf("%s: owner_sources = %v, want ok false naming it", kind, s)
}
for name, row := range out["peers"].(map[string]map[string]any) {
o := row["owner"].(map[string]any)
if o["source"] != "unread" || !strings.Contains(o["reason"].(string), kind) {
t.Fatalf("%s: %s owner = %v, want unread naming the failed source", kind, name, o)
}
}
}

// A failure in a layer the owner tiers do not read leaves the owners read.
mesh := []access.Note{{Kind: "mesh_layer_unavailable", Severity: access.SeverityError}}
if s := peersAnswer(src, mesh, []string{"machine-anvil-pod-7"})["owner_sources"].(map[string]any); s["ok"] != true {
t.Fatalf("mesh failure: owner_sources = %v, want ok", s)
}
}
Loading
Loading