Bring in desk from news
The plugin contract, the model task and the tools roledesk imports, copied rather than reached through an editable install of news, which also exposes a package named pipeline. news stays the source of truth for desk. Refreshed from news/desk at a275c38a1aa0fe80a7ec7dd5781ecccf683f71c6 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
1 parent
9d666d0937
commit
248ea1636d
36 files changed
+8393
No files matched your search
@@ -0,0 +1,103 @@
|
||||
"""Video, read as text with timings on it.
|
||||
|
||||
The generic half of watching things so you don't have to. Give it a video and
|
||||
it gives back a list of `Segment`s — each one a stretch of the recording with a
|
||||
title, the words spoken in it, and a link that plays it from that second.
|
||||
|
||||
read() a video -> Video + Segment[] the whole thing, one call
|
||||
youtube captions, metadata, comments where both inputs come from
|
||||
marks an outline -> chapter marks description, notes, comments
|
||||
media a MediaRef -> a playable link YouTube today, a file later
|
||||
segment cues + marks -> Segment[] the cut, and transcript text
|
||||
|
||||
**This package knows nothing about newsdesk.** No store, no queue, no config,
|
||||
no `Item`. That is what makes it reusable, and it is the seam the design is
|
||||
built on: `desk/sources/podcast.py` sits on top and adds the things that
|
||||
*are* specific to a show — its feed, its weekly cadence, the article links in
|
||||
its notes, and turning segments into rows.
|
||||
|
||||
Two decisions worth knowing before reading further.
|
||||
|
||||
**A video is a `MediaRef`, not a URL.** Today every ref is a YouTube id; the
|
||||
reading pane's link is computed from it on the way out. Point one at a file on
|
||||
disk in config and every link to it changes, with nothing rewritten and no
|
||||
item's identity touched. See `media.py`.
|
||||
|
||||
**Transcripts come from whoever can supply one.** `youtube-transcript-api`
|
||||
first, and a built-in fetcher against YouTube's own player endpoint when the
|
||||
library can't get the track. Transcribing audio locally (yt-dlp plus
|
||||
faster-whisper) is the third backend and the one that makes local video work;
|
||||
it is not written yet, and `read` is shaped so that adding it changes nothing
|
||||
above.
|
||||
|
||||
Try it:
|
||||
|
||||
python -m desk.tools.video https://www.youtube.com/watch?v=GUYEofBebDc
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
from desk.tools.video import marks, media, youtube
|
||||
from desk.tools.http import session
|
||||
from desk.tools.video.marks import Chapter, Link
|
||||
from desk.tools.video.media import MediaRef
|
||||
from desk.tools.video.segment import Segment, Video, cut, opening, paragraphs
|
||||
from desk.tools.video.youtube import Blocked, Cue, TranscriptUnavailable
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
__all__ = [
|
||||
"Blocked", "Chapter", "Cue", "Link", "MediaRef", "Segment",
|
||||
"TranscriptUnavailable", "Video", "cut", "describe", "marks", "media",
|
||||
"opening", "paragraphs", "read", "youtube",
|
||||
]
|
||||
|
||||
|
||||
def read(url: str, *, comments: bool = True,
|
||||
client: httpx.Client | None = None) -> tuple[Video, list[Segment]]:
|
||||
"""One video, cut into its chapters. The whole package in one call.
|
||||
|
||||
`comments` is the only cost worth thinking about: it is an extra request to
|
||||
the watch page and is only consulted when the description has no outline in
|
||||
it, so for a show that publishes chapter marks it never happens.
|
||||
"""
|
||||
with session(client, timeout=40) as http:
|
||||
video = describe(url, client=http)
|
||||
try:
|
||||
cues = youtube.transcript(video.key, client=http)
|
||||
except TranscriptUnavailable as exc:
|
||||
# Not fatal on its own. The outline still gives every chapter, its
|
||||
# links and its timecode; only the quoted words are missing, and
|
||||
# auto-captions usually appear within a few hours of an upload.
|
||||
log.warning("%s", exc)
|
||||
cues = []
|
||||
|
||||
chapters = marks.best(video.notes)
|
||||
if not chapters and comments:
|
||||
chapters = marks.in_comments(youtube.comments(video.key, client=http))
|
||||
|
||||
return video, cut(video, cues, chapters)
|
||||
|
||||
|
||||
def describe(url: str, *, client: httpx.Client | None = None) -> Video:
|
||||
"""What a video says about itself: title, author, length, description.
|
||||
|
||||
YouTube only, for now. A local file would come back with its stem as the
|
||||
title and no notes, which is why `Video.notes` is allowed to be empty and
|
||||
why `read` has a comments fallback at all.
|
||||
"""
|
||||
key = youtube.video_id(url)
|
||||
with session(client) as http:
|
||||
meta = youtube.details(key, client=http)
|
||||
return Video(
|
||||
ref=media.youtube(key),
|
||||
title=meta["title"],
|
||||
url=media.youtube(key).at(0),
|
||||
author=meta["author"],
|
||||
duration=meta["duration"],
|
||||
notes=meta["description"],
|
||||
)
|
||||
Reference in new issue
Block a user