A distributed system for processing astronomical spectral data using containerised scientific applications on Kubernetes/OpenShift.
ac3_astroapp/
├── astroapp/ # Backend processing system and scientific applications
├── connector/ # Eclipse Dataspace Connector for federated data transfer
├── experiment-tool/ # Experiment orchestration and metrics collection
└── gui/ # React web interface
The core processing backend. Contains:
- Go-based microservices (producer, consumer, processor)
- RabbitMQ-based job distribution
- S3 storage integration
- Scientific processing applications (Starlight, pPXF, Voronoi)
Eclipse Dataspace Connector (EDC) implementation for IONOS S3. Enables federated data transfer between provider and consumer buckets with bidirectional synchronisation.
Go CLI tool for orchestrating processing experiments. Manages dataset submission, monitors S3 for completion, and collects timing metrics for analysis.
React-based web interface for:
- Dataset management and file uploads
- Processing pipeline monitoring
- Aladin sky viewer integration
- Result visualisation
Each component has its own deployment configuration for OpenShift/Kubernetes. See individual component READMEs for specific instructions.
- OpenShift/Kubernetes cluster
- Container registry access (quay.io)
- S3-compatible storage (IONOS Cloud)
- RabbitMQ
-
Deploy infrastructure (namespace, volumes, RabbitMQ):
cd astroapp/application/deployments oc apply -f namespace.yaml oc apply -f volume.yaml -f volumeclaim.yaml oc apply -f rabbitmq.yaml -
Deploy backend services:
oc apply -f producer.yaml oc apply -f processor.yaml oc apply -f consumer.yaml
-
Deploy frontend:
cd ../../gui make rebuild
Environment variables are defined in deployment YAML files.