Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,11 @@
This document summarizes notable updates since February 2025, with commit dates from the repository history for context.【728428†L1-L48】

## 2026-07
- **UX (2.1.5):** The "unclear sentence" report control on english-to-LTL questions no longer reads as a heading for the answer options. It was a bold rust question ("Is this English sentence confusing or unclear?") sitting an equal distance from the stem above and the radio list below, so it grouped with neither; its `.row.ml-2` wrapper also pulled it left of the stem's text edge, since Bootstrap's `.row` sets `margin-left: -15px` and `ml-2` only partly cancels it. It is now a quiet caption-style link reading "Report unclear wording" (imperative, so it cannot be mistaken for the question to answer), tucked under the sentence with asymmetric spacing (about 4px above, 18px below) that groups it with the stem. Being quiet means losing the color signal, so it carries a permanent underline rather than color alone (WCAG 1.4.1), and uses `--ink-2` at 8:1 rather than the muted `--ink-3`, which is 4.17:1 on the card and under AA for small text; color returns on hover and focus. The control stays a sibling of `.actualQuestion`, so its label never leaks into the logged `question_text`. The modal, its route, and its payload are unchanged.
- **A11y (2.1.5):** The keyboard focus ring now actually appears on that control. The theme's global ring is a zero-specificity `:where(...):focus-visible` rule, which Bootstrap's `.btn:focus { outline: 0 }` outranks on any button, so the only focus signal would have been a color shift; `.btn.unclear-flag:focus-visible` restates it at (0,3,0). The letter key also uses `--ink-2` (8:1) rather than `text-muted` (`--ink-3`, 4.17:1 on the card, under AA at that size), since the key is the only place the letters in the options are defined. The rest of the page still pairs `text-muted` with `small` at that sub-AA ratio (card-header meta, question description); that is pre-existing and wants its own pass.
- **Bugfix (2.1.4):** Themed english-to-LTL questions now state what their letters mean. The sentence was in words ("the document is open") while every answer option was in letters (`d`, `c`), and nothing connected the two, so a student also had to guess that `d` names the document being *open*, not the document, which has more than one state. Each themed question now carries a key listing only the literals its formula uses (`d`: the document is open), rendered above the sentence. Themed responses logged before this fix measured LTL reading confounded with guessing the naming, and both themed arms are affected; per-arm analyses spanning the change should be segmented. The abstract control arm is untouched, since it quotes its literals in the prose already and there is nothing to look up.
- **UX (2.1.4):** A deontic english-to-LTL question now asks its scenario in the question prompt rather than stacking it above the sentence: the card header reads "You are auditing access to a confidential document. Which of the following LTL formulae best represents this policy?", built from the theme's `preamble` and new `rule_noun` fields. Previously the scenario and a "Company policy:" lead-in were glued onto the front of `question` with newlines and rendered at heading size, giving the framing the same visual weight as the property to formalize. The card body is now the letter key plus the one sentence to formalize. Other arms keep the generic "…best represents this English sentence?" prompt. `question` is the sentence alone, and the logged `question_text` is the key plus that sentence; the scenario is recoverable from `translation_mode`, which already identifies the arm.
- **UX (2.1.4):** The footer's green "Distinguished Paper at CAV 2025" button, shown on every page of the tutor, is now a plain "CAV 2025 Paper" text link alongside Blog Post / Brown PLT / GitHub, pointing at the same paper. (The README, which addresses researchers rather than students, still names the award.)
- **UX (2.1.3):** Submitting the "Is this English sentence confusing or unclear?" report on an english-to-LTL question now acknowledges the feedback instead of silently closing the modal: a persistent note appears under the question ("Thank you for your feedback. Please answer with your best-guess interpretation of the sentence."), announced to screen readers via `role="status"`. A failed submission now alerts the student instead of logging only to the console.
- **UX (2.1.2):** Misconception explainers now show inline LTL operators and formulas in the learner's selected Classic, Forge, or Electrum syntax, updating immediately when the top-bar syntax choice changes. Explainer fragments are authored in Classic inside `<code class="ltlformula">` and rewritten client-side by token substitution — safe because the tutor grammar only permits lowercase atoms, so a bare `X`/`F`/`G`/`U` is always an operator.
- **Bugfix (2.1.2):** The precedence explainer claimed `a | b & c -> d` parses as `((a | b) & c) -> d`; the parser actually yields `(a | (b & c)) -> d` (binary operators bind `&` > `|` > `U` > `->` > `<->`, left-associative within a level). Separately, the Electrum rendering of `U` formulas rendered operands with the Forge renderer, so e.g. `(X p) U q` displayed `NEXT_STATE` where Electrum uses `AFTER`.
Expand Down
30 changes: 26 additions & 4 deletions src/exercisebuilder.py
Original file line number Diff line number Diff line change
Expand Up @@ -489,16 +489,17 @@ def gen_nl_question_contextualized(self, formula, theme_name="lights"):
"""Generate a contextualized English question using the given theme.

Expects the formula's literals to already match the theme (see
gen_contextualized_answer). Deontic themes get their stance-setting
preamble prepended. Returns None if translation fails.
gen_contextualized_answer). Returns the rule sentence alone: a
deontic theme's stance-setting preamble is returned separately by
build_english_to_ltl_question, which hands it to the UI as part of
the question prompt rather than the property to formalize. Returns
None if translation fails.
"""
theme = ltltoeng_contextualized.THEMES[theme_name]
as_node = ltlnode.parse_ltl_string(formula)
result = ltltoeng_contextualized.translate(as_node, theme)
if not result or result.strip() == "":
return None
if theme.preamble:
result = f"{theme.preamble}\n\n{result}"
return result


Expand Down Expand Up @@ -638,13 +639,31 @@ def build_english_to_ltl_question(self, answer, contextualized=False, theme_name
if theme_name is None and contextualized:
theme_name = "lights"

preamble = ""
rule_noun = ""
legend = []
if theme_name is not None:
question = self.gen_nl_question_contextualized(answer, theme_name)
translation_mode = self.THEME_TRANSLATION_MODES[theme_name]
# Fall back to abstract if contextualized translation fails
if question is None or question == "":
question = self.gen_nl_question(answer)
translation_mode = "abstract"
else:
theme = ltltoeng_contextualized.THEMES[theme_name]
preamble = theme.preamble
rule_noun = theme.rule_noun
## The key ships with the question, not just with the theme:
## a themed sentence is in words while its options are in
## letters, so posing one without the other would ask the
## student to guess the naming as well as read the LTL.
## Options are mutations of this formula and never introduce
## literals of their own, so the answer's literals cover them.
legend = [
{"literal": lit, "meaning": phrase}
for lit, phrase in ltltoeng_contextualized.legend(
ltlnode.parse_ltl_string(answer), theme)
]
else:
question = self.gen_nl_question(answer)
translation_mode = "abstract"
Expand All @@ -655,6 +674,9 @@ def build_english_to_ltl_question(self, answer, contextualized=False, theme_name

return {
"question": question,
"preamble": preamble,
"rule_noun": rule_noun,
"legend": legend,
"type": self.ENGLISHTOLTL,
"options": options,
"translation_mode": translation_mode
Expand Down
52 changes: 49 additions & 3 deletions src/ltltoeng_contextualized.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@
Public API
----------
translate(node, theme=None) -> str
legend(node, theme=None) -> [(literal, state phrase)]
remap_to_theme(node, theme=None) -> node or None
THEMES: dict of built-in theme names -> Theme objects

Expand All @@ -34,10 +35,20 @@
state where it is true, not only on a false->true transition, so edge
phrasing would describe a different formula than the one being asked.

The sentence itself never mentions the letters, but the answer options are
written in them, so a themed question must be posed together with its key
(`legend()`): otherwise the exercise also tests whether the student can guess
that `d` names "the document is open" -- and not, say, the document, which has
more than one state -- which is a guessing game about naming, not about LTL.
A key entry names a whole state, never a subject, because that is what a
literal denotes.

Example:
node = parse_ltl_string("G(b -> F a)")
translate(node, theme=THEMES["lights"])
# => "Whenever the blue light is on, then eventually the amber light is on."
legend(node, theme=THEMES["lights"])
# => [("b", "the blue light is on"), ("a", "the amber light is on")]
"""

from __future__ import annotations
Expand Down Expand Up @@ -65,17 +76,27 @@ class Theme:
description: One-line description of the scenario.
literals: Maps literal names to (positive_phrase, negative_phrase).
e.g. {"b": ("the blue light is on", "the blue light is off")}
The positive phrase doubles as the literal's key entry
(see legend), so it must name the state the letter stands
for on its own, out of sentence context.
deontic: Phrase the formula as an enforceable rule ("must") rather
than a description. Requires copular literal phrases
("the X is Y") so the modal transform stays grammatical.
preamble: Stance-setting text shown before the sentence (only
meaningful for deontic themes). Empty = no preamble.
preamble: Stance-setting sentence putting the student in the role of
enforcing the rule (only meaningful for deontic themes).
Empty = no preamble.
rule_noun: What the sentence *is*, as a noun phrase, completing
"...best represents ___?". Empty falls back to the
generic "this English sentence". Pairs with preamble:
both belong to the question being asked, not to the
property, so the UI puts them in the question prompt.
"""
name: str
description: str
literals: Dict[str, tuple[str, str]] # lit -> (positive, negative)
deontic: bool = False
preamble: str = ""
rule_noun: str = ""

def positive(self, lit: str) -> str:
if lit in self.literals:
Expand Down Expand Up @@ -114,7 +135,13 @@ def negative(self, lit: str) -> str:
name="Document Access Audit",
description="Auditing access to a confidential document.",
deontic=True,
preamble="You are auditing access to a confidential document. Company policy:",
# Both of these ride in the question prompt ("You are auditing access to a
# confidential document. Which of the following LTL formulae best
# represents this policy?") rather than above the sentence: the stance is
# part of what is being asked, and hoisting it keeps the formalizable rule
# the single prominent line in the card body.
preamble="You are auditing access to a confidential document.",
rule_noun="this policy",
literals={
"d": ("the document is open", "the document is closed"),
"c": ("the user's clearance is active", "the user's clearance is revoked"),
Expand Down Expand Up @@ -638,6 +665,25 @@ def translate(node: ltlnode.LTLNode, theme: Optional[Theme] = None) -> str:
return result


def legend(node: ltlnode.LTLNode, theme: Optional[Theme] = None) -> list[tuple[str, str]]:
"""The key for *node*: (letter, state it stands for) pairs, theme order.

The sentence is in words and the answer options are in letters, so the
student needs the correspondence stated rather than inferred from initials
("d" is the document being *open*, not the document). Only literals the
formula actually uses are listed, so the key never hints at attributes the
question does not involve.

Entries are the plain *state* phrases even for a deontic theme: a letter
denotes a state of the world, and the obligation belongs to the policy
being asked about, not to the letter.
"""
if theme is None:
theme = THEMES["lights"]
used = collect_literals(node)
return [(lit, theme.literals[lit][0]) for lit in theme.literals if lit in used]


def collect_literals(node: ltlnode.LTLNode) -> set:
"""Return the set of literal names appearing in *node*."""
if isinstance(node, ltlnode.LiteralNode):
Expand Down
32 changes: 32 additions & 0 deletions src/static/css/theme.css
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,38 @@ code, pre, .mono, kbd { font-family: var(--font-mono); }
.btn-link { color: var(--accent-strong); font-weight: 600; }
.btn-link:hover { color: var(--accent); }

/* Passive escape hatch under an English stem: deliberately quiet, since a
student only reaches for it when stuck. Underlined rather than colored:
--ink-3 is only 4.17:1 on --surface (under AA for small text), and color
alone would violate WCAG 1.4.1. Color returns on hover/focus.
.btn.unclear-flag (0,2,0) outranks .btn's 700 and .btn-link's 600 (0,1,0)
whatever the source order, so no !important is needed. */
.unclear-flag-row { margin-top: .25rem; margin-bottom: 1.15rem; }
.btn.unclear-flag {
color: var(--ink-2);
font-weight: 400;
font-size: .8125rem;
line-height: 1.4;
text-decoration: underline;
text-decoration-color: var(--line-2);
text-underline-offset: 2px;
}
.btn.unclear-flag:hover, .btn.unclear-flag:focus {
color: var(--accent-strong);
text-decoration-color: currentColor;
}
/* The global focus ring is a :where() rule and so carries zero specificity;
Bootstrap's .btn:focus { outline: 0 } outranks it on any button. Restate it
here at (0,3,0) so keyboard users get the ring, not just a color shift. */
.btn.unclear-flag:focus-visible { outline: 3px solid var(--focus); outline-offset: 2px; }
.unclear-ack { color: var(--ink-2); font-size: .8125rem; margin-top: .25rem; }

/* Scenario + letter key above an English stem. Subordinate to the rule, but
--ink-2 (8:1) rather than the text-muted --ink-3 (4.17:1, under AA here):
the key is the only place the letters in the options are defined, so it has
to stay legible. */
.question-setup { color: var(--ink-2); font-size: .9375rem; line-height: 1.55; }

/* ---- Cards: flat, hard-edged, structural borders ------------------------ */
.card {
background: var(--surface);
Expand Down
41 changes: 27 additions & 14 deletions src/templates/exercise.html
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,9 @@
</div>
<p class="mb-0 font-weight-bold">
{% if q.type == "englishtoltl" %}
Which of the following LTL formulae best represents this English sentence?
{# A themed arm folds its scenario into the prompt, so the card body is
left with just the key and the one sentence to formalize. #}
{% if q.preamble %}{{ q.preamble }} {% endif %}Which of the following LTL formulae best represents {{ q.rule_noun | default('this English sentence', true) }}?
{% elif q.type == "tracesatisfaction_mc" %}
Which of the following traces satisfies the following LTL formula?
{% elif q.type == "tracesatisfaction_yn" %}
Expand Down Expand Up @@ -77,21 +79,32 @@ <h5 id="{{ qindex }}question_text" class="card-title mb-2">
<pre><code class="actualQuestion language-ltl">{{ q.question}}</code></pre>
</h5>
{% else %}
<h5 id="{{ qindex }}question_text" class="card-title mb-2 actualQuestion"
style="white-space: pre-wrap;"
data-translation-mode="{{ q.translation_mode | default('') }}">{{ q.question}}
</h5>
<div class="row mb-2 ml-2">
{% if q.type == "englishtoltl" %}
<button type="button" class="btn btn-link text-primary p-0" data-toggle="modal" data-target="#ltlToEngRaterModal-{{ qindex }}">
Is this English sentence confusing or unclear?
</button>
<p id="unclear-ack-{{ qindex }}" class="w-100 text-muted mb-0 mt-1 d-none" role="status">
<small>Thank you for your feedback. Please answer with your best-guess interpretation of the sentence.</small>
</p>
{% include 'ltltoengrater.html' %}
{# englishtoltl tightens the gap below the stem so the report control that
follows groups with the sentence rather than heading the answer options. #}
<div id="{{ qindex }}question_text"
class="actualQuestion {% if q.type == 'englishtoltl' %}mb-0{% else %}mb-2{% endif %}"
data-translation-mode="{{ q.translation_mode | default('') }}">
{% if q.legend %}
<ul class="list-unstyled question-setup question-legend mb-2">
{% for entry in q.legend %}
<li><code class="language-ltl">{{ entry.literal }}</code>: {{ entry.meaning }}</li>
{% endfor %}
</ul>
{% endif %}
<h5 class="card-title mb-0" style="white-space: pre-wrap;">{{ q.question }}</h5>
</div>
{% if q.type == "englishtoltl" %}
<div class="unclear-flag-row">
<button type="button" class="btn btn-link unclear-flag p-0"
data-toggle="modal" data-target="#ltlToEngRaterModal-{{ qindex }}">
Report unclear wording
</button>
<p id="unclear-ack-{{ qindex }}" class="unclear-ack mb-0 d-none" role="status">
Thank you for your feedback. Please answer with your best-guess interpretation of the sentence.
</p>
</div>
{% include 'ltltoengrater.html' %}
{% endif %}
{% endif %}

<ul class="list-group">
Expand Down
7 changes: 2 additions & 5 deletions src/templates/footer.html
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,9 @@
});
</script>
<div class="container text-center py-2" id="about">
<div class="mb-2">
<a href="https://cs.brown.edu/~sk/Publications/Papers/Published/pgnk-ltl-tutor/paper.pdf" target="_blank" rel="noopener" class="btn btn-sm btn-outline-success">
Distinguished Paper at CAV 2025
</a>
</div>
<div class="d-flex justify-content-center align-items-center flex-wrap">
<a href="https://cs.brown.edu/~sk/Publications/Papers/Published/pgnk-ltl-tutor/paper.pdf" target="_blank" rel="noopener" class="mx-2 text-muted">CAV 2025 Paper</a>
<span class="text-muted">·</span>
<a href="https://blog.brownplt.org/2024/08/08/ltltutor.html" target="_blank" rel="noopener" class="mx-2 text-muted">Blog Post</a>
<span class="text-muted">·</span>
<a href="https://cs.brown.edu/research/plt/" target="_blank" rel="noopener" class="mx-2 text-muted">Brown PLT</a>
Expand Down
2 changes: 1 addition & 1 deletion src/templates/version.html
Original file line number Diff line number Diff line change
@@ -1 +1 @@
2.1.3
2.1.5
Loading
Loading