Improved Youtube RSS/Atom feed

Last modification on 2023-11-20

... improved at least for my preferences ;)

It scrapes the channel data from Youtube and combines it with the parsed Atom feed from the channel on Youtube.

The Atom parser is based on sfeed, with some of the code removed because it is not needed by this program. It scrapes the metadata of the videos from the channel its HTML page and uses my custom JSON parser to convert the Javascript/JSON structure.

This parser is also used by the json2tsv tool. It has few dependencies.

Features

Add the video duration to the title to quickly see how long the video is.
Filter away Youtube shorts and upcoming videos / announcements: only videos are shown.
Supports more output formats: Atom, JSON Feed or sfeed Tab-Separated-Value format.
Easy to build and deploy: can be run as a CGI program as a static-linked binary in a chroot.
Secure: additionally to running in a chroot it can use pledge(2) and unveil(2) on OpenBSD to restrict system calls and access to the filesystem.

How to use

There is an option to run directly from the command-line or in CGI-mode. When the environment variable $REQUEST_URI is set then it is automatically run in CGI mode.

Command-line usage:

youtube_feed channelid atom
youtube_feed channelid gph
youtube_feed channelid html
youtube_feed channelid json
youtube_feed channelid tsv
youtube_feed channelid txt

CGI program usage:

The last basename part of the URL should be the channelid + the output format extension. It defaults to TSV if there is no extension. The CGI program can be used with a HTTPd or a Gopher daemon such as geomyidae.

For example:

Atom XML:     https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.xml
HTML:         https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.html
JSON:         https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.json
TSV:          https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.tsv
twtxt:        https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.txt
TSV, default: https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw

Gopher dir:   gopher://codemadness.org/1/feed.cgi/UCrbvoMC0zUvPL8vjswhLOSw.gph
Gopher TSV:   gopher://codemadness.org/0/feed.cgi/UCrbvoMC0zUvPL8vjswhLOSw

An OpenBSD httpd.conf using slowcgi as an example:

server "codemadness.org" {
	location "/yt-chan/*" {
		request strip 1
		root "/cgi-bin/yt-chan"
		fastcgi socket "/run/slowcgi.sock"
	}
}

Using it with sfeed

sfeedrc example of an existing Youtube RSS/Atom feed:

# list of feeds to fetch:
feeds() {
	# feed <name> <feedurl> [basesiteurl] [encoding]
	# normal Youtube Atom feed.
	feed "yt IM" "https://www.youtube.com/feeds/videos.xml?channel_id=UCrbvoMC0zUvPL8vjswhLOSw"
}

Use the new Atom feed directly using the CGI-mode and Atom output format:

# list of feeds to fetch:
feeds() {
	# feed <name> <feedurl> [basesiteurl] [encoding]
	# new Youtube Atom feed.
	feed "idiotbox IM" "https://codemadness.org/yt-chan/UCrbvoMC0zUvPL8vjswhLOSw.xml"
}

... or convert directly using a custom connector program on the local system via the command-line:

# fetch(name, url, feedfile)
fetch() {
	case "$1" in
	"connector example")
		youtube_feed "$2";;
	*)
		curl -L --max-redirs 0 -H "User-Agent:" -f -s -m 15 \
			"$2" 2>/dev/null;;
	esac
}

# parse and convert input, by default XML to the sfeed(5) TSV format.
# parse(name, feedurl, basesiteurl)
parse() {
	case "$1" in
	"connector example")
		cat;;
	*)
		sfeed "$3";;
	esac
}

# list of feeds to fetch:
feeds() {
	# feed <name> <feedurl> [basesiteurl] [encoding]
	feed "connector example" "UCrbvoMC0zUvPL8vjswhLOSw"
}

Screenshot using sfeed_curses

Clone

git clone git://git.codemadness.org/frontends

Browse

You can browse the source-code at:

The program is: youtube/feed

Dependencies

C compiler.
LibreSSL + libtls.

Build and install

$ make
# make install

That's all

I hope by sharing this it is useful to someone other than me as well.