Bring in desk from news
The plugin contract, the model task and the tools roledesk imports, copied rather than reached through an editable install of news, which also exposes a package named pipeline. news stays the source of truth for desk. Refreshed from news/desk at a275c38a1aa0fe80a7ec7dd5781ecccf683f71c6 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9d666d0937
commit
248ea1636d
36 files changed
+8393
No files matched your search
@@ -0,0 +1,546 @@
|
||||
"""A news podcast, read as one item per story rather than one per episode.
|
||||
|
||||
The general half. `desk/tools/video/` below knows how to cut a video into
|
||||
chapters; this knows the things that are true of a *show*: it has a feed, it
|
||||
publishes on a cycle, its episodes are somewhere on YouTube, and its stories
|
||||
belong in your queue next to the RSS copies of those same articles.
|
||||
`plugins/lastweekin_ai.py` above knows one show.
|
||||
|
||||
That three-layer split is not decoration. A show that publishes a timestamped
|
||||
outline works here with nothing show-specific involved — Dwarkesh and Latent
|
||||
Space both do. What varies between shows is smaller than it looks, and is
|
||||
declared as two methods rather than guessed at:
|
||||
|
||||
video_key(entry) where this show puts its video. The default reads
|
||||
the feed body, which is the common case and is
|
||||
wrong for several of the biggest shows.
|
||||
episode_number(title) how this show writes an episode number.
|
||||
|
||||
Where each piece comes from, for one episode:
|
||||
|
||||
the feed title, date, the audio enclosure, the video link,
|
||||
and the show notes as HTML — one fetch, everything
|
||||
the notes chapter marks, sometimes with the article links attached,
|
||||
which is what makes a segment a story rather than a topic
|
||||
YouTube the transcript, with timings
|
||||
video.cut() one Segment per chapter
|
||||
this file one Item per Segment
|
||||
|
||||
**The subscription is just a feed.** `feeds:` is the same config key `rss`
|
||||
uses, so a show imports into the `subscriptions` table like any other
|
||||
subscription and gets add/remove/fetch/show, the Sources tab and the sidebar
|
||||
for free — none of which had to know podcasts exist.
|
||||
|
||||
Config, for any show that is not one somebody has written a plugin for:
|
||||
|
||||
sources:
|
||||
podcast:
|
||||
feeds:
|
||||
- url: https://api.substack.com/feed/podcast/69345.rss
|
||||
label: Dwarkesh
|
||||
episodes_per_feed: 2
|
||||
|
||||
Two things this deliberately does not do. It does not summarise: `text` starts
|
||||
as the opening of the transcript and is replaced when a summary lands. What the
|
||||
hosts made of a story is the one thing here that no feed already gives you, and
|
||||
so the only part of this worth a model — which makes it a separate job, whose
|
||||
failure costs you the abstracts and not the pull. See
|
||||
newsdesk/jobs/summarise.py.
|
||||
|
||||
And it does not store the episode as an item, or as anything else: an episode
|
||||
is a date and a video id, both of which live on every segment it produced, so
|
||||
"every segment of episode 255, in order" is already a query and needs no
|
||||
entity behind it.
|
||||
|
||||
**A show with no outline gets nothing stored, on purpose.** That claim above
|
||||
used to hold only by luck: with no marks anywhere, `cut` hands back the whole
|
||||
episode as one segment — right for a link you added by hand, wrong here — and
|
||||
this stored it, so a two-hour recording arrived as a single eleven-thousand-word
|
||||
item whose standfirst was the first two sentences of the intro and which then
|
||||
queued itself for a summariser expecting one story. Now `_one_episode` stops
|
||||
before the transcript is even fetched and says so in the log. Only shows that
|
||||
publish a timestamped outline work here, which was always true and is now also
|
||||
enforced.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
from collections.abc import Iterator
|
||||
from datetime import UTC, datetime, timedelta
|
||||
from typing import Any
|
||||
|
||||
import feedparser
|
||||
import httpx
|
||||
|
||||
from desk.sources.base import USER_AGENT, CheckResult, Item, Source, entry_time
|
||||
from desk.sources.rss import check_feed
|
||||
from desk.sources.spec import (
|
||||
API_VERSION, RECORDING_READER, RSS, PluginSpec, Profile, RetryPolicy, Step,
|
||||
SummaryPolicy, TaggingPolicy, TargetSpec, Tasks,
|
||||
)
|
||||
from desk.tools.http import session
|
||||
from desk.tools.video import Segment, Video, cut, marks, media, opening, youtube
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: "LWiAI Podcast #255 - Gemini 3.7, ..." -> 255. Every show numbers its
|
||||
#: episodes somewhere; the title is where they all agree to put it.
|
||||
_NUMBER = re.compile(r"(?:#|episode\s+|ep\.?\s*)(\d{1,5})\b", re.IGNORECASE)
|
||||
|
||||
|
||||
def segment_profile(instructions: str = "", tags: str = "",
|
||||
target_words: int = 70) -> Profile:
|
||||
"""What a show that publishes a timestamped outline produces, declared.
|
||||
|
||||
A reusable typed preset, so that a plugin for one show is its feed, its
|
||||
numbering and its editorial line — and not a second copy of this
|
||||
machinery's description. `plugins/lastweekin_ai.py` is the model.
|
||||
|
||||
The two steps are what `_one_episode` below does, in the contract's words:
|
||||
retrieve the recording's captions *once*, then cut them at the marks the
|
||||
show published. Nothing executes them yet — the runner is a later step of
|
||||
`docs/plugin-rework.md` — and they are written now because they are the
|
||||
part a future reader has to be able to check this file against.
|
||||
|
||||
`instructions` is the one thing a show is expected to fill in: what its
|
||||
own segments want said about them. What is general about summarising
|
||||
speech — write what they made of it, keep the disagreement, do not quote a
|
||||
mangled caption back — belongs to the shared task and is written once in
|
||||
`newsdesk/jobs/summarise.py`, not per show.
|
||||
"""
|
||||
return Profile(
|
||||
# One profile, because one thing is stored: the stories, not the
|
||||
# episodes they came out of. An episode is a date and a video id, both
|
||||
# of which are on every segment it produced.
|
||||
kind="segment",
|
||||
acquisition=[
|
||||
Step(id="captions", tool="retrieve_captions",
|
||||
inputs={"media": "inputs.media"},
|
||||
outputs={"transcript": "inputs.transcript"},
|
||||
# Auto-captions on a two-hour upload can take hours to
|
||||
# appear, so absence is a wait and not a failure. Today that
|
||||
# wait is "the next pull, for ever", because the episode is
|
||||
# simply not marked done; a bound is what this declares.
|
||||
retry=RetryPolicy(max_attempts=4, delay_seconds=3600,
|
||||
backoff=2.0, on=["not_ready"])),
|
||||
Step(id="segments", tool="cut_segments", needs=["captions"],
|
||||
# The marks come from the show's own notes, which arrive in
|
||||
# the feed entry — HTML, so each story keeps the article link
|
||||
# beside its timestamp.
|
||||
inputs={"transcript": "inputs.transcript",
|
||||
"marks": "inputs.feed_text"},
|
||||
outputs={"transcript": "inputs.transcript",
|
||||
"links": "inputs.links"}),
|
||||
],
|
||||
tasks=Tasks(
|
||||
summary=SummaryPolicy(
|
||||
enabled=True, input_fields=["inputs.transcript"],
|
||||
require_complete=True,
|
||||
# No floor: a two-sentence housekeeping segment gets one flat
|
||||
# sentence saying so, which is worth having and is what the
|
||||
# shared prompt already asks for.
|
||||
min_words=0, target_words=target_words,
|
||||
instructions=instructions),
|
||||
tagging=TaggingPolicy(
|
||||
# The transcript, not the abstract: tagging a segment must not
|
||||
# quietly wait on a summariser that failed. Naming it decides
|
||||
# *which* text and not how much of it — the tagger reads the
|
||||
# opening few hundred characters of whatever a profile names,
|
||||
# because it is choosing from a closed list of labels and a
|
||||
# two-hour transcript would be thirty times the tokens for
|
||||
# the same answer. See `newsdesk/jobs/tagging.py`.
|
||||
enabled=True, input_fields=["inputs.transcript"],
|
||||
context_fields=["title", "inputs.links"], instructions=tags),
|
||||
),
|
||||
# The player first, then what the hosts made of it; the articles
|
||||
# they were reading stay above it in every view. See the preset.
|
||||
presentation=RECORDING_READER,
|
||||
)
|
||||
|
||||
|
||||
class PodcastSource(Source):
|
||||
name = "podcast"
|
||||
description = "A show's feed, cut into one item per story discussed."
|
||||
subscription_field = "feeds"
|
||||
default_outlet = "Podcast"
|
||||
|
||||
#: The generic source declares the same profile it hands its subclasses,
|
||||
#: with no editorial line of its own: it is any show, and it cannot know
|
||||
#: what one of them wants said. Its targets are editable because they are
|
||||
#: whatever you put under `podcast:` in config.yaml — a show with a plugin
|
||||
#: of its own declares a catalogue instead.
|
||||
spec = PluginSpec(
|
||||
api_version=API_VERSION, id=name, version="1", label="Podcast",
|
||||
discovery=RSS(),
|
||||
targets=TargetSpec(mode="editable", kind="rss_feed", defaults=[]),
|
||||
profiles={"default": segment_profile()},
|
||||
)
|
||||
|
||||
# -------------------------------------------------------- what a show is
|
||||
#
|
||||
# The two things that differ between shows, as methods rather than as
|
||||
# config knobs — because neither has a value you could write down. Finding
|
||||
# the video means knowing where that show puts it, and numbering an episode
|
||||
# means knowing how that show writes a number. A subclass that knows one
|
||||
# show answers both; everything else in this file is the same for all of
|
||||
# them.
|
||||
|
||||
def video_key(self, entry: Any) -> str:
|
||||
"""The YouTube id for this episode, or "" if there isn't one to find.
|
||||
|
||||
**The one place a show's publishing habits really bite.** The default
|
||||
reads the feed body, which works when the show embeds or links its own
|
||||
video there — Last Week in AI, Dwarkesh, Latent Space all do.
|
||||
|
||||
It is not general, and it fails in a way worth naming: The Vergecast,
|
||||
Hard Fork, TWiT, All-In and Lex Fridman all publish full video to
|
||||
YouTube and none of them mentions it in the feed, so all five come back
|
||||
empty here. Fixing that means matching a feed entry against a channel's
|
||||
uploads — which title, which date, whose channel — and that is a
|
||||
per-show judgement, not a parser. This is where it goes when someone
|
||||
wants one of those shows: override this, return the id, and every other
|
||||
line in this file already works.
|
||||
"""
|
||||
try:
|
||||
return youtube.video_id(_body(entry))
|
||||
except ValueError:
|
||||
return ""
|
||||
|
||||
def episode_number(self, title: str) -> int | None:
|
||||
"""Which episode this is, from its title, or None if it doesn't say.
|
||||
|
||||
Every show numbers its episodes and the title is where they all agree
|
||||
to put it — but not in one format. `#255` and `Episode 42` and `Ep. 7`
|
||||
are the common family and are what the default reads; `TWiT 1100` and
|
||||
`SN 1094` are not, and an interview show may not number at all.
|
||||
|
||||
Nothing downstream depends on the answer: it is stored on the item for
|
||||
display and for "every segment of episode 255, in order". So the honest
|
||||
default is a regex that catches most of them and returns None rather
|
||||
than guessing when it doesn't, and a subclass overrides it when its
|
||||
show writes numbers its own way.
|
||||
"""
|
||||
found = _NUMBER.search(title or "")
|
||||
return int(found.group(1)) if found else None
|
||||
|
||||
# ------------------------------------------------------------------ fetch
|
||||
|
||||
def fetch(self) -> Iterator[Item]:
|
||||
limit = int(self.options.get("episodes_per_feed", 2))
|
||||
with session(None, timeout=40) as http:
|
||||
for entry in self.options.get("feeds", []):
|
||||
url = entry["url"] if isinstance(entry, dict) else entry
|
||||
label = _label(entry, url)
|
||||
try:
|
||||
yield from self._one_feed(http, url, label, limit)
|
||||
except youtube.Blocked:
|
||||
# The one failure that is not this show's. YouTube refused
|
||||
# the machine, so every other show here would be refused
|
||||
# too, and each attempt is one more request from an address
|
||||
# already taken for a bot. Out of `fetch` altogether: the
|
||||
# runner keeps what was yielded and turns this into a
|
||||
# notice, which is what the log line below never was.
|
||||
raise
|
||||
except Exception:
|
||||
# One dead show must not cost the others their episodes.
|
||||
log.exception("podcast feed %s failed", url)
|
||||
|
||||
def _one_feed(self, http: httpx.Client, url: str, label: str,
|
||||
limit: int) -> Iterator[Item]:
|
||||
response = http.get(url, headers={"User-Agent": USER_AGENT})
|
||||
response.raise_for_status()
|
||||
# The bytes, not `.text`. A feed declares its own encoding in its XML
|
||||
# declaration and plenty of them are not UTF-8; handing the parser a
|
||||
# string that httpx has already decoded by its own guess throws that
|
||||
# declaration away, and what comes back is an episode title with a
|
||||
# mojibake apostrophe in it. `check_feed` parses the bytes too.
|
||||
feed = feedparser.parse(response.content)
|
||||
if not feed.entries:
|
||||
log.warning("no episodes in %s", url)
|
||||
return
|
||||
|
||||
for entry in feed.entries[:limit]:
|
||||
episode = self._episode(entry, label)
|
||||
if episode is None:
|
||||
continue
|
||||
if self._already_done(episode.key):
|
||||
log.debug("%s: already ingested", episode.title)
|
||||
continue
|
||||
try:
|
||||
yield from self._one_episode(http, episode, entry, label)
|
||||
except youtube.Blocked:
|
||||
raise # not this episode's failure — see `fetch`
|
||||
except Exception:
|
||||
log.exception("episode %s failed; continuing the feed", episode.key)
|
||||
|
||||
def _one_episode(self, http: httpx.Client | None, episode: Video, entry: Any,
|
||||
label: str) -> Iterator[Item]:
|
||||
# The show's own notes first: same outline as the video description,
|
||||
# but the links survived. Then the description, then the comments.
|
||||
chapters = marks.best(episode.notes)
|
||||
if not chapters:
|
||||
described = youtube.details(episode.key, client=http)
|
||||
chapters = marks.best(described["description"])
|
||||
episode.duration = episode.duration or described["duration"]
|
||||
if not chapters and self.options.get("read_comments", True):
|
||||
chapters = marks.in_comments(youtube.comments(episode.key, client=http))
|
||||
|
||||
if not chapters:
|
||||
# Nothing published an outline: not the notes, not the description,
|
||||
# not a listener in the comments. `cut` would hand back the whole
|
||||
# episode as one segment — correct of it, and what a hand-added
|
||||
# link wants — but this source's contract is one item per *story*,
|
||||
# and quietly substituting one item per episode is a different
|
||||
# product wearing its name.
|
||||
#
|
||||
# It is also unusable downstream in a way that is easy to miss. The
|
||||
# item's `text` would be `opening(transcript)`, the first two
|
||||
# sentences of a two-hour recording, which is what the ranker and
|
||||
# the tag filters would then judge it on; and eleven thousand words
|
||||
# would go to a summariser whose prompt promises it a single story
|
||||
# with the article the hosts were reading. Better to store nothing and say
|
||||
# why than to store one wall and three wrong things about it.
|
||||
#
|
||||
# Marked done regardless. Reaching here means the episode was
|
||||
# fetched and read successfully; there was simply nothing to cut
|
||||
# along. A show that never publishes an outline would otherwise
|
||||
# re-download a full transcript on every pull, for ever, to throw
|
||||
# it away again — which is the waste `_already_done` exists to stop.
|
||||
log.warning(
|
||||
"%s: no timestamped outline in the notes, the description or "
|
||||
"the comments, so there are no stories to cut it into — "
|
||||
"storing nothing for this episode", episode.title)
|
||||
self._mark_done(episode.key)
|
||||
return
|
||||
|
||||
try:
|
||||
cues = youtube.transcript(episode.key, client=http)
|
||||
except youtube.Blocked:
|
||||
# A `TranscriptUnavailable` too, and the one that must not be
|
||||
# treated as one here. Waiting is the answer to captions that are
|
||||
# still being made; to a refusal it is the same nothing on every
|
||||
# pull, at INFO, with the episode never marked done and nobody
|
||||
# told. Still not marked done, so the pull after the address is
|
||||
# clean reads it — but it leaves as a failure, and reaches the
|
||||
# screen. `details` above raises the same thing the same way.
|
||||
raise
|
||||
except youtube.TranscriptUnavailable as exc:
|
||||
# Not done, so no marker is written and the next pull tries again.
|
||||
# Auto-captions on a two-hour upload can take hours to appear, and
|
||||
# an episode without them is worth waiting for rather than
|
||||
# ingesting as twenty titles with nothing under them.
|
||||
log.info("%s: %s", episode.title, exc)
|
||||
return
|
||||
|
||||
segments = cut(episode, cues, chapters)
|
||||
log.info("%s: %d segments", episode.title, len(segments))
|
||||
for segment in segments:
|
||||
yield _item(segment, entry, label,
|
||||
number=self.episode_number(episode.title))
|
||||
self._mark_done(episode.key)
|
||||
|
||||
def check(self) -> list[CheckResult]:
|
||||
"""A show is a feed, so the feed check is the right one."""
|
||||
return [check_feed(url, label) for url, label in self.targets()]
|
||||
|
||||
def _episode(self, entry: Any, show: str) -> Video | None:
|
||||
"""A feed entry as a `Video`. None when there is no video to read.
|
||||
|
||||
A show whose video this cannot find is not a failure — see
|
||||
`video_key`, which is the part a subclass replaces. Saying so once,
|
||||
quietly, beats failing per episode.
|
||||
"""
|
||||
key = self.video_key(entry)
|
||||
if not key:
|
||||
# Deliberately not "audio-only shows need local transcription",
|
||||
# which is what this used to say and which was wrong for the five
|
||||
# biggest shows it fires on: they all post video, they just do not
|
||||
# say so in the feed. Naming the actual condition is what stops the
|
||||
# next person going and building a transcriber they did not need.
|
||||
log.info("no video id in the feed entry for %r — the show may still "
|
||||
"publish one; see PodcastSource.video_key",
|
||||
(entry.get("title") or "")[:60])
|
||||
return None
|
||||
|
||||
return Video(
|
||||
ref=media.youtube(key),
|
||||
title=(entry.get("title") or "").strip(),
|
||||
url=entry.get("link") or "",
|
||||
author=entry.get("author") or show,
|
||||
published_at=entry_time(entry),
|
||||
notes=_body(entry),
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------ bookkeeping
|
||||
|
||||
def _already_done(self, key: str) -> bool:
|
||||
"""Whether this episode has been through here successfully.
|
||||
|
||||
In the `state` table rather than in a table of its own, because that is
|
||||
genuinely all the state an episode has: it is done or it is not. An
|
||||
episode is not a thing you read, so it needs no row of its own, and one
|
||||
key in a table that already exists needs no migration.
|
||||
|
||||
The key is written only on success, so an episode whose captions
|
||||
YouTube had not finished generating is left unmarked and comes round
|
||||
again on the next pull.
|
||||
|
||||
Without this, a weekly pull would re-fetch and re-cut every transcript
|
||||
in the window on every run and then discard the lot at the dedup check
|
||||
— correct, and several wasted seconds per pull.
|
||||
"""
|
||||
return bool(self.state.get(self._done_key(key)))
|
||||
|
||||
def _mark_done(self, key: str) -> None:
|
||||
self.state[self._done_key(key)] = datetime.now(UTC).isoformat()
|
||||
|
||||
def _done_key(self, key: str) -> str:
|
||||
"""Namespaced by the plugin, not hard-coded to `podcast`.
|
||||
|
||||
Two shows can publish the same video id — a guest episode cross-posted,
|
||||
a network re-run — and with one shared prefix the second would find the
|
||||
first's marker and skip an episode it had never read. One subclass per
|
||||
show means one namespace per show for free.
|
||||
"""
|
||||
return f"{self.name}:done:{key}"
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ internals
|
||||
|
||||
def _label(entry: Any, url: str) -> str:
|
||||
"""What one configured show is called. Which outlet that stands for is the
|
||||
subscription's business — see `Item.subscription`."""
|
||||
if not isinstance(entry, dict):
|
||||
return url
|
||||
return entry.get("label") or url
|
||||
|
||||
|
||||
def _item(segment: Segment, entry: Any, label: str,
|
||||
number: int | None = None) -> Item:
|
||||
"""One segment, as the row newsdesk stores.
|
||||
|
||||
Three of the field choices are not obvious:
|
||||
|
||||
`body` is the transcript, which is both where it belongs and how this
|
||||
tells `fetch_bodies` not to go and scrape a YouTube page for one. It
|
||||
is not the `excerpt`, because the excerpt is what the queue row renders,
|
||||
what the ranker's document is built from and what search indexes, and
|
||||
sixteen hundred words of speech would swamp all three.
|
||||
|
||||
`excerpt` is a placeholder until a summary is written over it. It is the
|
||||
opening of the transcript rather than empty, because an item with no
|
||||
excerpt is an empty row in the queue, and the ranker and the tagger both
|
||||
read it. It used to be accompanied by a `summarised: False` flag, which
|
||||
was this source telling the summariser that the row was waiting for it.
|
||||
Nothing needs telling now: the profile declares that these items are
|
||||
summarised, and an empty `summary` column is the whole of "not yet".
|
||||
|
||||
`published_at` is the episode's stamp plus the segment's offset, so twenty
|
||||
items published in the same instant still sort in the order they were said.
|
||||
|
||||
`links` is the reason a segment is a story rather than a topic: the
|
||||
articles the outline attached to this stretch, typed and roled, so the
|
||||
reading pane and the summariser read them from the same field an outlet's
|
||||
links live in rather than from this plugin's own corner of `raw`.
|
||||
"""
|
||||
episode = segment.video
|
||||
return Item(
|
||||
source_id=segment.external_id,
|
||||
location=segment.url(),
|
||||
title=segment.title,
|
||||
excerpt=opening(segment.transcript),
|
||||
body=segment.transcript,
|
||||
author=episode.author,
|
||||
published_at=episode.published_at + timedelta(seconds=segment.start),
|
||||
subscription=label,
|
||||
links=segment.reading(),
|
||||
# Not `article`, which is what every one of these used to be stored as
|
||||
# — twenty-three transcripts filed as prose. What the pipeline made,
|
||||
# not what carried it: this arrived over RSS and left as segments.
|
||||
kind="segment",
|
||||
raw={
|
||||
# The chapter this segment sits under. Per *item* — it changes
|
||||
# down the episode — so it is here rather than on the
|
||||
# subscription's `section`, which names a branch of an outlet and
|
||||
# is the same for everything that subscription hands over.
|
||||
"heading": segment.heading,
|
||||
# Where it plays and what it linked to: the record every stored
|
||||
# segment carries, defined once on `Segment`.
|
||||
**segment.record(),
|
||||
"show": label,
|
||||
"episode": number,
|
||||
"episode_title": episode.title,
|
||||
"episode_url": episode.url,
|
||||
"episode_published": episode.published_at.isoformat(),
|
||||
"audio_url": _enclosure(entry),
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
def _body(entry: Any) -> str:
|
||||
"""The richest form of the notes the feed offers."""
|
||||
for content in entry.get("content") or []:
|
||||
if value := content.get("value"):
|
||||
return value
|
||||
return entry.get("summary") or ""
|
||||
|
||||
|
||||
def _enclosure(entry: Any) -> str:
|
||||
return next(
|
||||
(e.get("href", "") for e in entry.get("enclosures") or []
|
||||
if (e.get("type") or "").startswith("audio")),
|
||||
"",
|
||||
)
|
||||
|
||||
|
||||
# ----------------------------------------------------------------- dry run
|
||||
|
||||
def main(argv: list[str] | None = None) -> int:
|
||||
"""`python -m desk.sources.podcast <feed url>` — what a pull would store.
|
||||
|
||||
Runs the plugin with no store behind it and prints the items instead of
|
||||
writing them. This is the thing to run when a show changes the shape of
|
||||
its notes: everything else in the pipeline is downstream of these fields.
|
||||
"""
|
||||
import argparse
|
||||
import sys
|
||||
|
||||
parser = argparse.ArgumentParser(prog="python -m desk.sources.podcast")
|
||||
parser.add_argument("feed", help="the show's RSS feed")
|
||||
parser.add_argument("--episodes", type=int, default=1)
|
||||
parser.add_argument("-v", "--verbose", action="store_true")
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
logging.basicConfig(level=logging.INFO if args.verbose else logging.WARNING,
|
||||
format="%(levelname)-7s %(message)s")
|
||||
if hasattr(sys.stdout, "reconfigure"):
|
||||
sys.stdout.reconfigure(encoding="utf-8", errors="replace")
|
||||
|
||||
source = PodcastSource({
|
||||
"feeds": [args.feed], "episodes_per_feed": args.episodes,
|
||||
})
|
||||
items = list(source.fetch())
|
||||
if not items:
|
||||
print("no items — run with -v to see why", file=sys.stderr)
|
||||
return 1
|
||||
|
||||
print(f"{len(items)} items from {args.feed}\n")
|
||||
heading = None
|
||||
for item in items:
|
||||
if item.raw["heading"] != heading:
|
||||
heading = item.raw["heading"]
|
||||
print(f" {heading or '(unfiled)'}")
|
||||
stamp = marks.hhmmss(item.raw["start"])
|
||||
print(f" {stamp:>8} {item.title[:80]}")
|
||||
print(f" {item.location}")
|
||||
for link in item.links:
|
||||
print(f" → {link.url}")
|
||||
print(f" {len(item.body.split()):,} words · {item.excerpt[:90]}")
|
||||
print()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Reference in new issue
Block a user