Repository navigation
URL imports
#1775
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
URL Imports
It's useful to have imports that come from raw HTTPS urls for a number of reasons.
One particular use case that's come up recently is hosting files on Github: https://raw.githubusercontent.com/ urls are a useful place to host and fetch static code from a repository, which is already a good place to store code. This can be used to cut out publishing steps of going through Google Drive.
This proposal has a few steps aimed at different use cases and ergonomics
Raw
urlimportsAdd new syntax:
import url(https-url) as IThis expects:
This will be implemented with a basic
fetchcall from the browser/from the command line that does a simple HTTP CORS GET request to the URL with no special headers or preprocessing. The response is expected to by Pyret code (no importing raw JS via this mechanism). It is compiled and cached in memory (for CPO environments) and to disk incompiled/with a name based on the hash of its URL (for CLI environments). The identity of the module is the URL: if you import two modules with the same text with non-equal URLs, they are both instantiated separately. This does require a little care in not accidentally grabbing extra?or#fragments of URLs when copy/pasting, which could have spurious inequalities.url-fileimportsOne possible (and I hope to become common) workflow will be to author starter code and student libraries in VScode or in a Github Codespace, which has nice file management and version control built in along with the full-featured editor for reactors, etc.
One issue with authoring student-facing code meant to be used on code.pyret.org is that when publishing, it's useful to have a stable link in the import line. But while authoring, it's often useful to be able to make and test quick edits.
my-gdriveandshared-gdriveare clumsy for this because they require edits and refreshes. The analog for development withurl(...)in a codespace or vscode would be either:url. This requires pushing lots of little edits that are likely to have errors, and is cumbersome. Further, pushing to the canonical place of import could make users see these bugs.import file(...)while developing, and then switching back toimport url("https://raw.githubuerscontent...")for a push that publishes. If multiple files are involved and all of them need to be changed, it can be error-prone to push it out. Also it adds friction to making and testing a small edit.What's needed is an import form that can be stable in source, but do “the right thing” depending on its context. We propose adding
import url-file(base-url, rel-path) as IFor example:
The overall idea is that the URL is split into a base URL and a relative path fragment. The concatenation of them is used to fetch the URL when remote access is needed, while the relative path can be used in some development cases when the file is available locally on disk.
Along with this we introduce a new compiler option
--url-file-modewith three options:all-local,all-remote, andlocal-if-present.The semantics are:
url-fileimport always has an identity equal to the concatenation ofbase-urlandrel-path. This is important! It would be really weird if you could have the same URL present at two different “paths” on disk because it could lead to cases where either the module graph is different on CPO than on the command line, or there are conflicting sources of truth for the content of a file.url-filemodule:all-localmode, allurl-fileimports are treated asfileimports ofrel-path, except with module identity of the URL rather than the path on diskall-remotemode, allurl-fileimports are treated asurlimports with the URL made from concatenatingbase-urlandrel-pathlocal-if-presentmode, the compiler first checks if a file is present on disk at therel-path, in which case it loads it likefileimport, otherwise it fetches it remotely with the constructed URLThe default is always
all-remote. In CPO hosted at code.pyret.org there is no way to change the mode fromall-remote. At the CLI (e.g. pyret-npm) the behavior can be controlled with--url-file-mode. In the VScode extension there will be an extension and project setting to pick the mode (with the defaultall-remote).Here's what this enables:
Rachel is a Bootstrap curriculum author. She writes complex, multi-file support code for students using a Github Codespace in her fork of a curriculum repository called
rachel/curriculum. She toggles the mode tolocal-if-presentwhen she does her work. The import statements all look like:import url-file("https://raw.githubusercontent.com/bootstrapworld/curriculum/HEAD/...", "path/to/file.arr")As she edits files in her Codespace and re-runs, she sees all the changes immediately for testing and debugging. When she's happy with her code, she pushes it to her fork and submits a pull request to
bootstrapworld/curriculum.When the code merges, the import statements point to URLs hosted on Github that contain the current up-to-date copy of the code; the merge is a “deployment”.
Rachel's students do their work from code.pyret.org in single-file Pyret projects. They start their work from links like
https://code.pyret.org/editor#urlshare=https://raw.githubusercontent.com/bootstrapworld/curriculum/HEAD/, save copies of those files to their Google Drive, and always see the most up to date version of the curriculum.Kathi is an undergraduate instructor. All of her students work in single-file Pyret projects, and her TA team is used to making a single repository per assignment.
Her students use a mix of code.pyret.org and VScode based on their preferences and experience. Each repository has student starter code and some support files. Her TAs also write code using
url-file(...)imports.Some students do their work by forking the starter repository, cloning it, and opening it in VScode. If they want, the can see the library code provided in the project, but editing it doesn't change the behavior they see (because of the
all-remotesetting).Other students do their work by copy-pasting the single starter file into a new CPO window. The imports work because the refer to the Github static file URLs, and they don't need copies of all the files (just the one single-file project starter).
Other students do their work by copy-pasting the single starter file into a new .arr file in VScode. The imports work because the refer to the Github static file URLs, and they don't need copies of all the files (just the one single-file project starter).
Daniel is an undergraduate instructor. His students work on multi-file projects where they are expected to edit multiple files in a project directory, and may create new files. They also rely on some background code that they should not edit.
His TAs use
url-filefor the provided libary code that students shouldn't be able to edit. For the initial structure of the test and code files students see, which are intended to be editable by students, they use plainfileimports. If students add more student-authored files, they usefilefor these as well.Students are expected to use VScode or a Github Codespace to edit multi-file projects.
Some notes:
local-if-present, that is for TAs/curriculum authorsurlorurl-file. The main goal of this is to make it un-editable; students can easilycurlthe contents.js-fileimports (which is orthogonal to this infrastructure).All reactions