"""What a plugin declares about itself, as types rather than as convention. A plugin is still one file with a `Source` subclass in it — `registry.py` finds it and `base.py` is the contract for *fetching*. This is the other half, and it is declarative: instead of a plugin calling a model, cutting a transcript or emitting markup, it says which shared tools acquire its content, what the shared summariser and tagger should do with it, and how its fields are laid out. The application owns the doing. See `docs/plugin-rework.md`. class HackerNews(Source): name = "hacker_news" spec = PluginSpec(api_version=1, id=name, label="Hacker News", ...) Nothing executes a spec yet. This step introduces the contract and its validation so that later steps have one thing to implement against, rather than each inventing its own declaration and then reconciling them. Three rules hold this together: *Versioned.* `api_version` is the whole contract, not a part of it, which is why every type it needs is declared here at once. A field a later step starts *reading* costs nothing; a field a later step has to *add* means every plugin written against version 1 was written against a contract that did not say what it claimed to say. *Validated twice, for two different reasons.* What is wrong on its face — a step that needs one that isn't there, a cycle, a step writing over a summary — is a pydantic error raised where the plugin is declared, so a bad spec fails at import and `registry._load_file` logs it against that file and keeps every other plugin. What can only be judged by the app — is `transcribe_media` a tool this application has, is `segment` a kind it lays out — is `problems()`, because `desk` has no list of either and inventing one here would be the contract deciding what an app on it supports. See `Vocabulary`. *Undeclared is a legitimate state.* Every plugin that exists today declares no spec, and `spec_of` gives it one marked `adapted`: honest about the identity it does have and silent about everything it never said. An adapted spec is never invalid, because there is nothing in it that a plugin author wrote. """ from __future__ import annotations import re from dataclasses import dataclass from typing import Any, Literal from pydantic import model_validator from desk.models import Model from desk.tools import failures as failures_tool from desk.tools import marks as marks_tool #: The contract this module defines. A plugin naming any other number is not #: refused because it is wrong but because nothing here knows what it means. API_VERSION = 1 #: A plugin id, and a target id: stable machine names. Not the displayed #: outlet name — that is `label`, it is written for people, and it changes. #: An id that changed would orphan every stored item that recorded it. _MACHINE_NAME = re.compile(r"^[a-z][a-z0-9_]*$") #: Where an acquisition step is allowed to write. The plugin and its steps own #: inputs; the shared summariser and tagger own outputs. A step that could #: write `outputs.summary` would be a plugin generating its own summary #: through the back door, which is the one thing this contract exists to stop. _OWNED_BY_ACQUISITION = "inputs." #: The marks are a shared vocabulary with shared glyphs and shared filters; a #: plugin declares the *evidence* for one, never a mark of its own. _MARK_SLUGS = frozenset(marks_tool.BY_SLUG) #: What a failed step can be called. Shared for the same reason the marks are: #: the categories are what the runner *produces*, so a policy naming one they #: do not contain is a policy that matches nothing, and the place to find that #: out is the plugin file rather than an evening of retries that never happen. _FAILURES = failures_tool.CATEGORIES # --------------------------------------------------------------- acquisition class RetryPolicy(Model): """When a failed acquisition step is worth running again. Per step, not per plugin and not per pull: a transcript that was not ready an hour ago and an article whose host timed out are different waits, and retrying the whole outlet to get one of them is how a source ends up fetched four times an evening. """ #: Counting the first attempt, so 1 means "no retry" and reads as it says. max_attempts: int = 1 delay_seconds: int = 0 #: Multiplied into the delay for each further attempt. 1.0 is a fixed wait. backoff: float = 1.0 #: Which failures are worth another attempt — `timeout`, `server_error`. #: Empty retries none of them, which is what `max_attempts=1` already says. #: A consent wall is deliberately not in any app's list by default: it is #: the same answer however many times it is asked for. The names are #: `desk.tools.failures.CATEGORIES`; a misspelt one is refused here rather #: than becoming a policy that quietly matches nothing. on: list[str] = [] #: Whether the reader offers a person the retry this policy would not take #: on its own. manual: bool = False @model_validator(mode="after") def _bounded(self) -> RetryPolicy: if not 1 <= self.max_attempts <= 10: raise ValueError( f"max_attempts is {self.max_attempts}; it counts the first " "attempt, so it must be between 1 and 10") if self.delay_seconds < 0 or self.backoff < 1.0: raise ValueError("a retry delay cannot be negative and backoff " "cannot shrink it") unknown = sorted(set(self.on) - _FAILURES) if unknown: raise ValueError( f"retry on {', '.join(unknown)}: not a failure category " f"({', '.join(sorted(_FAILURES))})") return self class Step(Model): """One named piece of acquisition: a shared tool, and what it reads and writes. `inputs` maps the tool's own parameters to where their values come from; `outputs` maps what it produced to the item fields it lands in. Both sides named, so a reader of the plugin can see what a step will change without knowing the tool, and so the runner can refuse a step that would write somewhere it does not own. """ id: str tool: str inputs: dict[str, str] = {} outputs: dict[str, str] = {} #: Step ids in the same profile that must succeed first. The extraction of #: an article needs the destination URL a discovery hook worked out. needs: list[str] = [] #: A field path that must have a value for this step to run at all. Empty #: runs it always. when: str = "" retry: RetryPolicy = RetryPolicy() @model_validator(mode="after") def _writes_only_inputs(self) -> Step: if not _MACHINE_NAME.match(self.id): raise ValueError(f"step id {self.id!r} is not a machine name") for produced, destination in self.outputs.items(): if not destination.startswith(_OWNED_BY_ACQUISITION): raise ValueError( f"step {self.id!r} writes {produced!r} to {destination!r}; " "an acquisition step may only write `inputs.*` — outputs " "belong to the shared summariser and tagger") return self class Discovery(Model): """How a plugin finds items in the first place. Separate from acquisition because it answers a different question: not "what is in this item" but "which items are there". `plugin` means the class's own `fetch`, which is what every plugin does today. """ tool: Literal["plugin", "rss", "api"] = "plugin" #: One fixed feed, for a plugin whose whole catalogue is a single address. #: A plugin with more than one declares them in `targets` instead, and the #: discovery helper reads them from there; two competing lists of feeds is #: the thing this must not become. feed_url: str = "" options: dict[str, Any] = {} class RSS(Discovery): """`discovery=RSS(feed_url=...)`, which is the common case spelled out.""" tool: Literal["plugin", "rss", "api"] = "rss" # ---------------------------------------------------------------- the tasks class SummaryPolicy(Model): """What the shared summariser should do with this profile's items. Structured parameters settle length and eligibility; `instructions` is editorial and cannot change the schema, the fields written, or whether an item is eligible at all. It is added to the same model call rather than being a second pass over the answer. """ #: Required, and required even when false: a profile that says nothing #: about summarising is a profile whose author has not decided, and the #: cost of guessing is a model bill or a queue of items silently skipped. enabled: bool #: In order; the first field with usable content is the one summarised. #: Not concatenated — the same words twice is not more context. input_fields: list[str] = [] #: Whether a partial article — a feed's first paragraph — may be summarised #: as though it were the piece. require_complete: bool = True #: Below this, the item is skipped with a reason rather than summarised. #: A long article and a two-minute segment want different floors. min_words: int = 0 target_words: int = 140 instructions: str = "" @model_validator(mode="after") def _has_something_to_read(self) -> SummaryPolicy: return _needs_inputs(self, "summary") class TaggingPolicy(Model): """What the shared tagger should do with this profile's items. The vocabulary stays central: a plugin adds emphasis, never a tag namespace of its own. Prefer acquired content as the input rather than the summary, so tagging does not silently wait on a summary that failed. Naming a field says *which* text is read and never how much of it. An app that picks from a fixed list of labels may read the opening of it and no more — newsdesk does, at four hundred characters — so naming a transcript here is not a request to send a transcript. """ enabled: bool input_fields: list[str] = [] #: Read for context but not as the thing being tagged: the title, the #: publisher's own categories. context_fields: list[str] = [] instructions: str = "" @model_validator(mode="after") def _has_something_to_read(self) -> TaggingPolicy: return _needs_inputs(self, "tagging") def _needs_inputs[T: SummaryPolicy | TaggingPolicy](policy: T, name: str) -> T: """An enabled task with no input field is a task that will look at every item and find nothing to read — reported here rather than as an empty result for the whole archive.""" if policy.enabled and not policy.input_fields: raise ValueError(f"{name} is enabled but names no input_fields") return policy class Tasks(Model): """The shared tasks, both declared explicitly. See `SummaryPolicy.enabled` for why neither has a default.""" summary: SummaryPolicy tagging: TaggingPolicy # ---------------------------------------------------------- how it is shown class Section(Model): """One band of a view: a standard field, bound to a shared component. Not markup. Ordering the list is what moves a player above or below the links, and that is the whole of the layout language on purpose — safe links, responsive behaviour, loading and error states belong to the renderer, and every plugin that could express them would express them differently. """ component: str field: str label: str = "" #: For a `links` section: show only links with this role — `reading`, #: `discussion`, `source`. role: str = "" #: For a `media` section: which media reference, when an item has several. media: str = "" collapsed: bool = False class View(Model): """One of the reader's exclusive views, as this profile fills it in.""" label: str sections: list[Section] = [] #: Field paths that must have a value for this view to be offered. A #: Summary view with no summary is disabled and says why, rather than #: rendering an empty panel or a placeholder dressed as an abstract. requires: list[str] = [] #: What to show here when `requires` cannot be met: another view, with its #: own label, its own requirement and its own sections. The control keeps #: its place in the drawer and renames itself to say what is actually in #: it, which is the difference between a reader who sees the feed's own #: two sentences and one who sees a greyed-out button. #: #: The queue has done this since it existed — `QueueLayout.preview` is an #: ordered list and the first non-empty field wins. A row that falls #: through to the feed's text while the pane behind it refuses to show #: anything is the same item answering two different ways. instead: View | None = None def chain(self) -> list[View]: """This view and every fallback under it, best first.""" return [self, *(self.instead.chain() if self.instead else [])] class QueueLayout(Model): """The row, which is a decision rather than a read: a title, the best preview available, and whatever metadata makes the decision.""" #: In order, first non-empty wins. A generated summary, then the feed's #: own text — and a placeholder is never one of these. preview: list[str] = [] metadata: list[str] = [] #: Whether the row names the host its item points at, beside the outlet. #: True where the outlet is not the publisher — an aggregator's front page #: — because there the host is half of what the decision to read is made #: on, and "Hacker News" alone does not say whether this is a kernel #: mailing list or a newspaper. False everywhere the two are the same #: name twice. destination: bool = False class OriginalView(Model): """The destination itself, embedded. Hidden on mobile, and never shown over an outlet that refuses framing however this is labelled.""" field: str label: str = "Original" #: The marks whose items this view is not offered for. Per item, because #: the mark is: a piece behind the paywall frames the paywall, and the #: piece beside it in the same feed frames the article. The layout does #: not change — the control stays where it is, greyed, with the reason on #: it — because which items are readable is not a fact about the outlet. #: An outlet where *nothing* is readable is a `Wall`, which is a different #: thing and has a different flow. Mark slugs, from the shared vocabulary. blocked_by: list[str] = ["paywalled"] @model_validator(mode="after") def _known_marks(self) -> OriginalView: unknown = sorted(set(self.blocked_by) - _MARK_SLUGS) if unknown: raise ValueError( f"blocked_by names {', '.join(repr(m) for m in unknown)}, " f"which is not among the shared marks " f"({', '.join(sorted(_MARK_SLUGS))})") return self class ExternalLink(Model): """The new-tab action beside the views. Not a view: it leaves.""" field: str label: str = "" class Presentation(Model): """Declared once per profile, never per stored item. The drawer is fixed and the content is not. Every item is read through the same four controls in the same order, whatever produced it: - `summary`, the short version: an abstract, or the feed's own teaser - `text`, the whole words: an article, a transcript, a post - `original`, the thing itself inside the pane: the page framed, or the recording playing - `external`, the way out to a new tab A plugin decides which of them it fills, what each is called, what sits in each and what each falls back to. It never adds a fifth: a reader moving from a newspaper to a podcast to a front-page link keeps one hand position, and a sticky choice of view means the same thing on every item. `header` is what holds whichever view is on: where an item points and where it is discussed. An aggregator's destination is not its own outlet, and a reader in the Text view of a Hacker News link is owed both names without leaving it. """ queue: QueueLayout = QueueLayout() #: Shown above the pane in every view, including one that cannot be #: shown. `links` sections only: `inputs.original_url` is the item's own #: destination, `inputs.links` with a `role` is the typed links. A section #: with nothing in it drops out, so a show whose notes link nothing has no #: header rather than an empty one. header: list[Section] = [] default_view: str = "" views: dict[str, View] = {} original: OriginalView | None = None external: ExternalLink | None = None @model_validator(mode="after") def _default_is_a_view(self) -> Presentation: if self.default_view and self.default_view not in self.views: raise ValueError( f"default_view is {self.default_view!r}, which is not one of " f"the declared views ({', '.join(sorted(self.views)) or 'none'})") return self @model_validator(mode="after") def _header_is_links(self) -> Presentation: wrong = sorted({s.component for s in self.header} - {"links"}) if wrong: raise ValueError( f"the header holds links, and this one declares " f"{', '.join(repr(c) for c in wrong)}; prose and media belong " "in a view") return self #: What an ordinary published article looks like in a reader, as a preset a #: plugin can name instead of restating. The shapes are the contract's; which #: of them an application actually renders is its own business, and #: `problems()` is where the two meet. #: #: Summary before Text is the order the shell has, not a preference: a reader #: who wants the piece presses Text, and the view they were last in is #: remembered. A profile that wants its own arrangement declares one rather #: than editing this. ARTICLE_READER = Presentation( queue=QueueLayout(preview=["outputs.summary", "inputs.feed_text"], metadata=["outlet"]), #: The page this item is, named by its host. What a reader in the Text #: view is owed without leaving it: which page these words came from. header=[Section(component="links", field="inputs.original_url")], default_view="text", views={ # An outlet that is never summarised, or one whose article we could # not fetch, still published two sentences in its feed. They are the # shortest version of the piece we have, which is what this control is # for, so it shows them rather than greying out — and says "RSS feed", # because they are the publisher's own teaser and calling a teaser a # summary is how a reader comes to think our summaries are terrible. "summary": View(label="Summary", requires=["outputs.summary"], sections=[ Section(component="prose", field="outputs.summary"), ], instead=View(label="RSS feed", requires=["inputs.feed_text"], sections=[ Section(component="prose", field="inputs.feed_text"), ])), "text": View(label="Text", requires=["inputs.article_text"], sections=[ Section(component="prose", field="inputs.article_text"), Section(component="links", field="inputs.links", label="Links"), ]), }, original=OriginalView(field="inputs.original_url"), external=ExternalLink(field="inputs.original_url"), ) #: What a social post looks like, as the preset a network names. A post is #: its own text and a link out of it: there is no article to extract and no #: abstract to wait for, so the profile declares one view rather than #: offering a Summary that could never become ready. #: #: `inputs.feed_text` is the post itself and not a teaser for something #: else — the same field an outlet's feed fills, read here as what the #: network published rather than as a stub of an article behind it. #: #: The article section below it is the link post: a post that pointed at #: something and had it fetched shows both, in the order they were written. #: It is optional and a post without one simply has one band — which is why #: `requires` names the post text and not the article. POST_READER = Presentation( # The same order every row is built in, and not a narrower claim: the # queue falls through an abstract to the feed's own text whatever the # profile, and a post declaring only the second half would be describing # its own policy rather than the queue. There is never a summary here to # come first. queue=QueueLayout(preview=["outputs.summary", "inputs.feed_text"], metadata=["outlet"]), #: The page this item is, named by its host. What a reader in the Text #: view is owed without leaving it: which page these words came from. header=[Section(component="links", field="inputs.original_url")], default_view="text", views={ "text": View(label="Post", requires=["inputs.feed_text"], sections=[ Section(component="prose", field="inputs.feed_text"), Section(component="prose", field="inputs.article_text", label="What it linked to"), Section(component="links", field="inputs.links", label="Links"), ]), }, original=OriginalView(field="inputs.original_url"), external=ExternalLink(field="inputs.original_url"), ) #: What an outlet we cannot read at all looks like: the text its feed gave us, #: and the way out to the page. The flow a profile declaring a `Wall` wants — #: there is no article to put under a Text control, no abstract will ever be #: written, and the destination frames the gate, so none of the three is #: offered. `ARTICLE_READER` with two permanently disabled buttons says the #: same thing, and says it again on every item for as long as the wall stands. #: #: The feed's text sits in the summary slot, where `ARTICLE_READER` puts it #: when an abstract is missing, so that the control is in the same place and #: keeps the same sticky choice as a reader moves between outlets. FEED_READER = Presentation( queue=QueueLayout(preview=["outputs.summary", "inputs.feed_text"], metadata=["outlet"]), #: The page this item is, named by its host. What a reader in the Text #: view is owed without leaving it: which page these words came from. header=[Section(component="links", field="inputs.original_url")], default_view="summary", views={ "summary": View(label="RSS feed", requires=["inputs.feed_text"], sections=[ Section(component="prose", field="inputs.feed_text"), Section(component="links", field="inputs.links", label="Links"), ]), }, external=ExternalLink(field="inputs.original_url"), ) #: What a link to somebody else's page looks like: an aggregator's front page, #: where the item is a pointer and the outlet is not the publisher. The views #: are `ARTICLE_READER`'s, because what is behind the link is an article, or #: fails to be one, like any other. The header is what differs: the host the #: link goes to and the thread it is discussed in, above every view — so both #: are there when the page would not extract, which is exactly when a reader #: needs them, and the Text view is the article and nothing else. LINK_READER = Presentation( queue=QueueLayout(preview=["outputs.summary", "inputs.feed_text"], metadata=["outlet"], destination=True), header=[ Section(component="links", field="inputs.original_url"), Section(component="links", field="inputs.links", role="discussion"), ], default_view="text", views={ "summary": View(label="Summary", requires=["outputs.summary"], sections=[ Section(component="prose", field="outputs.summary"), ], instead=View(label="RSS feed", requires=["inputs.feed_text"], sections=[ Section(component="prose", field="inputs.feed_text"), ])), "text": View(label="Text", requires=["inputs.article_text"], sections=[ Section(component="prose", field="inputs.article_text"), ]), }, original=OriginalView(field="inputs.original_url"), external=ExternalLink(field="inputs.original_url"), ) #: What a stretch of a recording looks like: the player's strip, then what #: was said. Summary, Transcript, Watch — the same three places as an article, #: under the names of what is in them. #: #: No fallback to `inputs.feed_text` under Summary, which is what an article's #: does: a segment's feed text is the *episode's* whole timestamped outline, #: so falling through to it would put the running order under every story cut #: out of it. The transcript is a view you choose and not the thing under the #: abstract, because it arrives as captions with the names mangled. #: #: The articles the hosts were reading are the header, so they stay in reach #: while the recording plays. A show whose notes link nothing has no header, #: and needs no second preset for it. RECORDING_READER = Presentation( queue=QueueLayout(preview=["outputs.summary", "inputs.feed_text"], metadata=["outlet", "inputs.media"]), header=[Section(component="links", field="inputs.links", role="reading")], default_view="summary", views={ "summary": View(label="Summary", requires=["outputs.summary"], sections=[ Section(component="media", field="inputs.media"), Section(component="prose", field="outputs.summary"), ]), "text": View(label="Transcript", requires=["inputs.transcript"], sections=[ Section(component="media", field="inputs.media"), Section(component="prose", field="inputs.transcript"), ]), }, # Watch, not Original: the destination is the recording, playing at this # segment's own offset. original=OriginalView(field="inputs.media", label="Watch"), external=ExternalLink(field="inputs.original_url"), ) # -------------------------------------------------------------- the targets class Target(Model): """One feed or one account, as the plugin declares it. `id` is the archive's name for it and never the URL or the label: a feed that moves, or a handle that renames itself, must stay the same thing it was, with its items, votes and read state on it. """ id: str label: str = "" #: The address, handle or query this target is. Empty for an editable #: target, which gets its locator when someone adds it. locator: str = "" section: str = "" profile: str = "default" #: What this target does before anyone touches the switches. Seeded once; #: a later reload must not reassert them over a choice someone made. fetch: bool = True show: bool = True class TargetSpec(Model): """Whether the plugin's targets are a fixed catalogue or a list a person adds to — a plugin capability, not a default for every outlet. A publisher declares its own feeds and there is no Add box: another masthead's feed is another plugin, with that publisher's own extraction, marks and retries. A social network declares no accounts and the box is the point. """ mode: Literal["catalog", "editable"] #: What one of these *is*: `rss_feed`, `account`. The app decides which #: types it can add and validate. kind: str defaults: list[Target] = [] #: Names the check a typed value must pass before it is accepted — #: `bluesky_handle`. Empty accepts anything the app's own type check does. validator: str = "" default_fetch: bool = True default_show: bool = True @model_validator(mode="after") def _ids_are_stable_and_unique(self) -> TargetSpec: seen = [t.id for t in self.defaults] if len(seen) != len(set(seen)): raise ValueError("two targets share an id: " f"{sorted({i for i in seen if seen.count(i) > 1})}") for target in self.defaults: if not _MACHINE_NAME.match(target.id): raise ValueError(f"target id {target.id!r} is not a machine name") if self.mode == "catalog" and not self.defaults: raise ValueError("a catalog declares its targets, and this one " "declares none; use mode='editable' for a plugin " "whose targets are added by a person") return self # ----------------------------------------------------------------- the wall class Wall(Model): """What an outlet answers with instead of its articles — all of them. Not a mark, and the difference is the whole reason this exists. A mark is a fact about one item: this piece is behind the paywall, that one is an advert. The archive filters on it, refuses the embed for it, and leaves everything else about the outlet alone, because the piece next to it in the same feed is an ordinary readable article. A wall is a fact about the plugin: there is no article to fetch on any URL it produces, no abstract will ever be written from one, and the destination frames the gate rather than the piece. A walled outlet is not an outlet where everything is paywalled — it is one we cannot read at all, and what it needs is a different flow rather than the ordinary one with its controls greyed out on every item for ever. Declared rather than detected because the consequence is the *layout*. `desk.tools.walls` learns a wall from three identical bodies and stops fetching, which is right, is about a host, and arrives long after the reading pane has had to decide what to offer. A plugin that learns to get past its wall — a browser running the CMP's JavaScript for a tracking-consent gate — deletes this declaration and its profile is an ordinary article profile again, flow included. """ #: In the vocabulary the runner already reports failures in, so a declared #: wall and a detected one are the same word on the row: `consent_wall` #: for a tracking-consent gate, `denied` for a login or a bot check. kind: Literal["consent_wall", "denied"] #: Why, in words. For whoever reads the plugin, and for the reader, who is #: owed an answer to "where is the rest of this". reason: str = "" @model_validator(mode="after") def _says_why(self) -> Wall: if not self.reason.strip(): raise ValueError( "a wall needs a reason: it is what the reading pane tells " "somebody who can see there is no article here") return self # ---------------------------------------------------------------- the marks class MarkRule(Model): """How this outlet says "paid" or "advert" in its own text. The vocabulary and the glyphs stay shared; only where the evidence is found moves into the plugin. `strip` cuts a matched token out of the displayed title — the original is kept, because a title with its token already cut off cannot be reclassified later. """ #: Where to look: a field path, or a publisher category list. field: str method: Literal["token", "prefix", "suffix", "contains", "category"] value: str mark: str strip: bool = False @model_validator(mode="after") def _known_mark(self) -> MarkRule: if self.field not in {"title", "inputs.source_title", "inputs.feed_text", "inputs.source_categories"}: raise ValueError(f"unsupported mark input {self.field!r}") if not self.value.strip(): raise ValueError("a mark rule needs a non-empty value") categories = self.field == "inputs.source_categories" if categories != (self.method == "category"): raise ValueError("category matching requires source_categories") if self.strip and self.field not in {"title", "inputs.source_title"}: raise ValueError("only a title token may be stripped") if self.mark not in _MARK_SLUGS: raise ValueError( f"mark {self.mark!r} is not one of the shared marks " f"({', '.join(sorted(_MARK_SLUGS))}); a plugin declares the " "evidence for a mark, not a mark of its own") return self # ------------------------------------------------------------- the contract class Profile(Model): """One shape of content a plugin produces, end to end. A show that publishes both whole episodes and cut segments declares two of these rather than being two plugins, and each item records which one it is. That recorded name is how an item is laid out and processed for the rest of its life — never a guess made later from whether it happens to have a video key, or from what its outlet is called. """ #: `article`, `post`, `segment` in newsdesk. A free string here on #: purpose: which shapes exist is the app's business — see `Item.kind` — #: and the app's `Vocabulary` is what checks it. kind: str acquisition: list[Step] = [] #: None inherits the plugin rules; an explicit list replaces them. marks: list[MarkRule] | None = None #: What this outlet answers with instead of its articles, when it does. #: None is the ordinary case and says nothing; a `Wall` says there is no #: article behind any of these URLs, which is a different flow and not a #: worse version of this one. See `Wall`, and `FEED_READER` for the flow. #: #: The acquisition step stays declared beside it on purpose: it is what #: would run if the wall came down, and running it is how `desk.tools.walls` #: finds out that it has not. wall: Wall | None = None tasks: Tasks presentation: Presentation = Presentation() @model_validator(mode="after") def _a_wall_is_not_a_view(self) -> Profile: """A walled profile may not offer the destination as a view. The page frames perfectly well — a consent gate is a real page, and `framing` has no way to tell it from an article — so this is not something the app can find out for itself. And it is true of every item from this outlet, for as long as the wall stands, which makes it a layout decision rather than a control to grey out one item at a time. The way to the page is `ExternalLink`: in a tab of their own a reader can accept the gate and read the piece, which is the whole difference between useless and the way out. """ original = self.presentation.original if self.wall and original and original.field.split(".")[-1] != "media": raise ValueError( f"this profile declares a {self.wall.kind} and an Original " "view of the destination, which would frame the wall on every " "item; declare the way out as `external` instead — see " "`FEED_READER`") return self @model_validator(mode="after") def _acquisition_is_a_graph(self) -> Profile: steps = {s.id: s for s in self.acquisition} if len(steps) != len(self.acquisition): raise ValueError("two acquisition steps share an id") for step in self.acquisition: for needed in step.needs: if needed not in steps: raise ValueError( f"step {step.id!r} needs {needed!r}, which is not a " f"step of this profile ({', '.join(steps) or 'none'})") _no_cycles(steps) return self def _no_cycles(steps: dict[str, Step]) -> None: """Refuse a dependency loop where it is written rather than where it hangs. Depth-first with three colours: an edge back into the path being walked is the cycle, and the path is the useful half of the message.""" walking: list[str] = [] done: set[str] = set() def walk(current: str) -> None: if current in done: return if current in walking: loop = " -> ".join([*walking[walking.index(current):], current]) raise ValueError(f"acquisition steps depend on each other: {loop}") walking.append(current) for needed in steps[current].needs: walk(needed) walking.pop() done.add(current) for step_id in steps: walk(step_id) class PluginSpec(Model): """Everything a plugin declares. One version, one id, one set of profiles. Structural faults raise here, where the plugin is written. What only the application can judge is `problems()`. """ api_version: int #: Stable and machine-readable. Conventionally the class's `name`, which #: is already the config key and already on every stored item. id: str #: The plugin's own revision. Bumping it makes outputs produced under the #: old one recognisably stale without regenerating the archive. version: str = "1" #: What a person is shown. Free to change; `id` is not. label: str = "" discovery: Discovery = Discovery() targets: TargetSpec | None = None marks: list[MarkRule] = [] profiles: dict[str, Profile] #: True when nothing in the plugin file declared this and `spec_of` read #: it off the class. Such a plugin runs exactly as it did before the #: contract existed; nothing about it is a claim its author made. adapted: bool = False @model_validator(mode="after") def _coherent(self) -> PluginSpec: if self.api_version != API_VERSION: raise ValueError( f"plugin {self.id!r} declares api_version {self.api_version}; " f"this application implements version {API_VERSION}") if not _MACHINE_NAME.match(self.id): raise ValueError( f"plugin id {self.id!r} is not a machine name: lowercase " "letters, digits and underscores, starting with a letter") if not self.profiles: raise ValueError(f"plugin {self.id!r} declares no profiles; every " "plugin has at least a `default`") for name, target in _targets_of(self): if target.profile not in self.profiles: raise ValueError( f"target {name!r} names profile {target.profile!r}, which " f"this plugin does not declare " f"({', '.join(sorted(self.profiles))})") return self def _targets_of(spec: PluginSpec) -> list[tuple[str, Target]]: return [(t.id, t) for t in (spec.targets.defaults if spec.targets else [])] # ------------------------------------------------------- what the app allows @dataclass(frozen=True) class Vocabulary: """The names one application answers to, against which a spec is checked. Deliberately not a constant in this module. `desk` does not know which acquisition tools an app has wired up, which kinds it lays out, or which components it renders — and a list here would be this contract deciding what an application on it supports, which is the opposite of the point. An empty set means the app does not restrict that name, which is the right answer for `fields` until the typed item fields exist. """ tools: frozenset[str] = frozenset() kinds: frozenset[str] = frozenset() components: frozenset[str] = frozenset() views: frozenset[str] = frozenset() fields: frozenset[str] = frozenset() #: Target types the app can store and validate: `rss_feed`, `account`. target_kinds: frozenset[str] = frozenset() #: The named value checks the app implements for a target somebody types #: in — `bluesky_handle`. A plugin naming one the app has not written is #: asking for a guarantee nothing would enforce, which is worse than #: declaring none. Empty does not restrict it. validators: frozenset[str] = frozenset() #: The preview order the app's queue can actually produce, in order, with #: the leading group dropped. A queue row is one column in a list of #: hundreds, so what it can show is the app's to say — an app whose #: preview is a generated column can produce exactly one order, and a #: profile declaring another has to be told rather than quietly given the #: one order there is. Empty does not restrict it. queue_preview: tuple[str, ...] = () def problems(spec: PluginSpec, vocabulary: Vocabulary) -> list[str]: """Every reason this application cannot run this plugin, in one list. All of them, not the first: a plugin naming three tools an app does not have should say so once rather than over three edits. An empty list means it can be loaded. An adapted spec is always empty — see `spec_of`. Nothing in one was declared, so there is nothing in it to be wrong. """ if spec.adapted: return [] found: list[str] = [] for name, profile in sorted(spec.profiles.items()): where = f"{spec.id}.{name}" found += _unknown(vocabulary.kinds, [profile.kind], f"{where}: kind", "kinds this application lays out") found += _unknown(vocabulary.tools, [s.tool for s in profile.acquisition], f"{where}: acquisition tool", "tools it can run") found += _unknown( vocabulary.components, [s.component for view in profile.presentation.views.values() for shown in view.chain() for s in shown.sections] + [s.component for s in profile.presentation.header], f"{where}: component", "components it renders") found += _unknown(vocabulary.views, list(profile.presentation.views), f"{where}: view", "views its reader has") found += _unknown(vocabulary.fields, _fields_named(profile), f"{where}: field", "fields an item has") declared = tuple(name.split(".")[-1] for name in profile.presentation.queue.preview) if vocabulary.queue_preview and declared and declared != vocabulary.queue_preview: found.append( f"{where}: queue preview {' then '.join(declared)} is not an " f"order this application's queue can produce " f"({' then '.join(vocabulary.queue_preview)})") if spec.targets: found += _unknown(vocabulary.target_kinds, [spec.targets.kind], f"{spec.id}: target kind", "target types it can store") found += _unknown(vocabulary.validators, [v for v in [spec.targets.validator] if v], f"{spec.id}: target validator", "value checks it implements") return found def _unknown(allowed: frozenset[str], used: list[str], what: str, supported: str) -> list[str]: """One diagnostic per unrecognised name, naming what was expected. An empty `allowed` is "this app does not restrict these" and never a vocabulary in which every name is wrong.""" if not allowed: return [] return [f"{what} {name!r} is not among the {supported} " f"({', '.join(sorted(allowed))})" for name in sorted(set(used)) if name not in allowed] def _fields_named(profile: Profile) -> list[str]: """Every item field this profile binds, from any of the places one can be named. The leading group is dropped: `inputs.transcript` is the transcript wherever it is read from.""" named = [*profile.tasks.summary.input_fields, *profile.tasks.tagging.input_fields, *profile.tasks.tagging.context_fields, *profile.presentation.queue.preview, *profile.presentation.queue.metadata, *[section.field for section in profile.presentation.header]] for step in profile.acquisition: named += [*step.outputs.values(), *([step.when] if step.when else [])] for view in profile.presentation.views.values(): for shown in view.chain(): named += [section.field for section in shown.sections] named += shown.requires return [name.split(".")[-1] for name in named if name] # ------------------------------------------------ plugins that declare none def spec_of(source: type) -> PluginSpec: """The plugin's own spec, or one adapted from the class it already is. Every plugin in this codebase is undeclared today, and most will still be after the first few are converted, so this is the ordinary path and not a fallback. It reads only what the class actually states — its name, its description, and whether its subscriptions are feeds or handles — and states nothing else on the author's behalf. `adapted` is how everything downstream tells the difference between a plugin that declared no instructions and one that declared none *deliberately*. Read off the class itself and never off a parent: a show that subclasses `PodcastSource` is its own plugin with its own id and its own archive, and inheriting a declaration would quietly file its items under the class it borrowed from. A subclass that wants its parent's acquisition says so. The one profile it carries is `default` with both tasks disabled. That is not a claim that the plugin is not summarised or tagged — it is today, by the pipeline, exactly as before. It is this contract saying it has been told nothing about either, which is the truth. """ declared = source.__dict__.get("spec") if isinstance(declared, PluginSpec): return declared name = getattr(source, "name", "") or "" return PluginSpec( api_version=API_VERSION, id=name, version="0", label=getattr(source, "default_outlet", "") or name, profiles={"default": Profile( kind="", tasks=Tasks(summary=SummaryPolicy(enabled=False), tagging=TaggingPolicy(enabled=False)))}, adapted=True, )