Skip to content
USAREUR-AFPublic

About

goat: a self-hosted, open-source data fabric. WireGuard mesh (NetBird), brokerless Zenoh pub/sub and identity-based zero-trust access that keeps working when links fail. CLI, builder, roles and design record.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

22 stars

Watchers

1 watching

Forks

Latest commit

 

History

681 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

goat

goat

A self-hosted data fabric that keeps working when the network around it fails.

CI License: Apache 2.0

panoscope showing synthetic peers on a local Zenoh fabric, then the live stream of a newly published topic
panoscope, goat's live view of the data fabric, running on a laptop with synthetic data. Its code is in goatstuff.

Note

goat is pre-1.0: interfaces can still change between releases. To build on goat, use the stable branch or the goatcore/v tag it points at. To contribute, open a pull request against main. How the two fit together.

What goat is

goat connects people, machines and services across sites into one network, and lets them publish and subscribe to data with access decided by who they are, not where they sit. It is built for places where connectivity is poor or intermittent: once a device has joined, it keeps talking to its peers directly, with or without the cloud and with or without the central services that enrolled it.

goat is an integration of established open-source projects, assembled and operated as one system:

Concern Built on
Network WireGuard peer-to-peer tunnels, managed by NetBird, with relays for peers that cannot reach each other directly
Data Eclipse Zenoh publish/subscribe with no central broker
Identity Ory for people and services, SPIFFE for workloads, an offline certificate authority for machines
Site state PostgreSQL, replicated active-active between sites

On top of those, this repository provides two command-line tools: goatbuilder builds, releases and deploys a goat environment, and goat operates one.

What you can rely on

  • Nothing sensitive faces the internet. Every service listens only inside the encrypted network. The public internet sees DNS records, not open ports.
  • Joined devices keep working when the centre goes down. The management service is needed to enrol a device, not to keep it connected. If it fails, peers carry on talking to each other.
  • There is no central data broker to lose. Data moves between publishers and subscribers directly, so no single server stands between them.
  • Machines pull their configuration. Each machine fetches what it is meant to run and reports what it is running; a deployment is finished when every machine's report matches the declaration.
  • Every error says what to do next. Each message from goat and goatbuilder states what went wrong, why, and how to fix it; goat explain takes an error code or pasted output.

Try it

The fastest way to see goat is a rehearsal: goatbuilder stands up a complete example environment against a simulated set of machines, on your laptop, in seconds. Nothing touches a network. Download goatbuilder for Linux or macOS from the releases and run it from a clone of this repository:

git clone https://github.com/usareur-af/goatcore && cd goatcore

OS="$(uname -s | tr '[:upper:]' '[:lower:]')"; ARCH="$(uname -m | sed 's/x86_64/amd64/; s/aarch64/arm64/')"
curl -fsSL -o goatbuilder "https://github.com/usareur-af/goatcore/releases/download/goatbuilder/v0.7.1/goatbuilder-$OS-$ARCH"
chmod +x goatbuilder

WORK="$(mktemp -d)/zagros"
cp -R ops/goatbuilder/fixtures/goatnet-from-scratch/01-founder "$WORK"
./goatbuilder standup zagros --site alpha --mock "$WORK" --ack "you" --yes
./goatbuilder status --env zagros --mock "$WORK"

Each release lists a SHA-256 checksum per binary in SHA256SUMS, signed with cosign. To build it yourself instead, cd crates && cargo build --release -p goatbuilder puts it in crates/target/release/; the first build often takes half an hour.

The example environment, zagros, has one site with two core machines, one monitoring machine and one recovery machine. Tutorial 1 explains each step and carries on to real hardware.

To run goat for real on a single machine with no cloud account, start with goatlite.

To have a coding agent do it with you, give it this line with your own values: Clone https://github.com/usareur-af/goatcore and follow .claude/skills/goatlite/SKILL.md to stand up a goatlite named <name> on <ssh-user>@<host>, with <your-email> as its administrator.

Documentation

Learn goatlite: one machine, one file · A new environment · A second site · Sharing with a partner
Use Stand up a goatnet · The goat CLI · goatbuilder · Operations runbooks
Understand Architecture overview · Component designs · Decision record
This release What is included and what you provide

Related repositories

Repository Contents
goatenv A complete example environment, kept separate from the code it deploys
hello-galaxy-goatenv A step-by-step walkthrough that builds that environment
goatstuff Inspection tools and demonstration applications
goatdocs Architecture, strategy, design system and integration guides
zagros-moon-pod A pod that joins a goat environment, used today only to learn (ADR 1239)

Built on goat

Work that other people and organizations build on goat is listed in awesome-goat: open-source projects, commercial and proprietary products, feeds and integrations, deployments and research. If you have built something that uses goat as its substrate, add it there.

Repository layout

Path Contents
crates/ The goat and goatbuilder command-line tools and their shared libraries (Rust)
ops/ Ansible roles and playbooks, the portal and identity console, enrollment tooling, and the CI checks
goat-relay/, signal-server/ The relay and the signalling server
terraform/ Reusable cloud modules
docs/ Tutorials, runbooks, component designs and the decision record

Contributing

Pull requests are welcome. Read CONTRIBUTING.md before you start; every CI check is a script under ops/ci/ that you can run locally. Report security issues privately as described in SECURITY.md.

License

goat was written by U.S. Federal government employees. In the U.S. it is not subject to copyright and is in the public domain; in countries where copyright is recognized, it is released under the Apache License 2.0. See INTENT.md. Third-party components and their licences are listed in THIRD_PARTY_LICENSES.md; see also NOTICE.

About

goat: a self-hosted, open-source data fabric. WireGuard mesh (NetBird), brokerless Zenoh pub/sub and identity-based zero-trust access that keeps working when links fail. CLI, builder, roles and design record.

Topics

Resources

Code of conduct

Contributing

Security policy

Stars

22 stars

Watchers

1 watching

Forks

Releases

Packages

Used by

Contributors

Languages