[kasbah]

This is a complete re-work of the documents package, breaking with RFC-0006 significantly. It is comprised of:

(I am using model notebooks as the example here but this applies to instances too.)

1. Changes to the API

All changes to documents are now made with methods on the root document: add, update, and delete . update and delete take cell references (ids) in the case of notebooks, when updating cells.

data() returns a processed view of the notebook JSON.

The main reason for this change is mixing the view with mutation methods is inconvenient when trying to get a reactive view of the data.

Before

interface Notebook<S extends Shape> {
    document: ModelDocument;
    cells(): CellOf<S>[];
    cellsOf(typeOrShape: AnyCellType | Shape): CellOf<S>[];
    /* ... */
}

type CellOf<S extends Shape = Shape> =
    | RichTextCell
    | ObjectCell<ObjectTypesOf<S>>
    | MorphismCell<S, MorphismTypesOf<S>>
    | EquationCell<S>;

interface ObjectCell<O extends ObjectType> {
    kind: "object";
    id: string;
    type: O;
    label: string | undefined;
    update(patch: Partial<{ label: string | null }>): void;
    delete(): void;
}

After

type CellRef = { id: string };

interface Notebook<S extends Shape> {
    data(): ModelNotebookData;
    cells(): readonly CellData[];
    cellsOf(typeOrShape: AnyCellType | Shape): readonly CellData[];
    update(patch: Partial<{ title: string }>): void;
    update<C extends CellData>(cell: C, patch: CellPatchOf<S, C>): void;
    delete(cell: CellRef): void;
    /* ... */
}

interface ModelNotebookData {
    readonly type: "model";
    readonly name: string;
    readonly theory: string;
    readonly cells: readonly CellData[];
}

interface ObjectCellData<O extends ObjectType = ObjectType> {
    readonly generatorId: string; // Stored judgment ID, distinct from cell ID.
    readonly kind: "object";
    readonly id: string;
    readonly type: O;
    readonly label: string | undefined;
    /* No mutation methods! */
}

type CellData =
    | ObjectCellData
    | MorphismCellData
    | EquationCellData
    | RichTextCellData;

2. Changes to the internal workings of the documents package

We now snapshot the document and parse it into a known schema before we present that as the view. This aspect closes #1426. This also removes Automerge proxies from the view so we no longer have to worry about how to copy values.

We create a subscriptions when we create a document handle, that re-parses / validate the document on any changes. This means we need to dispose that subscription when we no longer immediatly care about that document.

The pipeline looks like this:

3. Cleaner separation of SolidJS concerns

We no longer inject SolidJS proxies via getDocumentView. Instead there is a new package catcolab-documents-solid that has some helper methods. The plan is to stop using solid-automerge and makeDocumentProjection altogether. We lose some fine-grained reactivity, in fact the current helpers are very coarse, but the fine-grained reactivity seemed like a premature optimization. We can apply more targeted optimizations to our new helpers as we need.

function createDocumentDataView<
    D extends {
        data(): unknown;
        onChange(callback: () => void): () => void;
    },
>(source: Accessor<D>): Accessor<ReturnType<D["data"]>>;

function createNotebookValidationView<S extends Shape>(
    notebook: Notebook<S>,
): Accessor<ModelValidation<S> | undefined>;

function createInstanceValidationView<H, S extends Shape, V>(
    instance: Instance<S, H, V>,
): Accessor<InstanceValidation<S> | undefined>;

[kasbah]

To answer two questions from IRL review with @tslil-topos

1. What does this TS parser check that the elaborator does not?

A: In the case of model notebooks, the structure parser is much more lenient. This part is mainly to avoid all the defensive programming down the line.

ModelNotebookData builder ("parse data" in the graph above) then brings everything into the shape we expect, keeping close to the fields outlined in the RFC, (whether that shouldn't already be the JSON shape is another question). It also links the references e.g. resolves an object for the from of a morphism.

2. What if we added a document type, do we have to add the parsing/normalization logic.

A: That depends on the format. There is no parsing logic for the llm-conversation, but I can imagine adding a format where some would be needed.