Skip to content
This repository has been archived by the owner on Dec 27, 2023. It is now read-only.
/ custom-rss Public archive

📡 Filtering RSS because Zapier is too expensive

License

Notifications You must be signed in to change notification settings

hlfcoding/custom-rss

Repository files navigation

Custom RSS

Build Status Code Climate Package License

📡 Filtering RSS because Zapier is too expensive (and Pipes is gone).

Features

  • Blacklist filtering
  • Repost guarding
  • Skipped entry logging
  • No database required
  • Uses JSON for configuration
  • Feeds as middleware functions
  • As few dependencies as possible
  • ES5, so compatible with older Node versions

Usage

The sample app is what's in the root directory. It's a barebones Connect app, with personal configuration in config.json. This is foremost for personal use, but what's in /src should be reusable with any Connect-like web framework. Or to use as-is:

$ npm run develop
$ subl config.json # continued below
$ subl src/feeds/feedd.js # continued below
$ git push origin master # git tag <tag> && git push origin <tag> && npm publish

$ ssh <production>
$ cd <site> # or `mkdir <site>; cd <site>`
$ git pull origin master # or `git clone <repo>`
$ npm start # or touch tmp/restart.txt
$ exit

$ curl <site-url>/feedd
// config.json
{
  "feeds": [
    // ...
    {
      "name": "feedd",
      "filters": [
        { "name": "tired-topics", "type": "blacklist", "tokens": [ "Foo", "Bar", "Baz" ] }
      ]
    }
  ]
}

// src/feeds/feedd.js
var fetchFeed = require('../fetch-feed');
var filterFeed = require('../filter-feed');
var url = require('url');

module.exports = function(config, request, response) {
  config.originalURL = 'http://feedd.com/rss.xml';
  config.url = url.format({
    protocol: 'http', host: request.headers.host, pathname: config.name
  });
  fetchFeed({
    url: config.originalURL,
    onResponse: function(resFetch, data) {
      response.setHeader('Content-Type', resFetch.headers['content-type']);
      filterFeed({
        config: config,
        data: data,
        onDone: function(data) { response.end(data); }
      });
    },
    onError: function(e) { response.end(e.message); }
  });
};

Feeds

  • Hacker News (via hnapp)
  • Quartz
  • NYTimes Business
  • Ray Wenderlich
  • Gama Sutra

Implementation

Some shared hosting providers, including mine, refuse to have NPM installed on their system. So dependencies need to be few to none, unless they're small enough to version. No XML parser or writer is used; a much lighter hand-rolled transformer does basic regex parsing. No MySQL client is used; data is stored with limits in plain files and manipulated in buffers (memory). No logger or mailer is used for feedback; custom loggers are handrolled as needed, with utilities on top of fs. The test-runner is handrolled (only because not a lot is required). Connect is the only dependency (but not really, see usage). This core constraint also yields the opportunity to learn Node fundamentals.

custom-rss

About

📡 Filtering RSS because Zapier is too expensive

Topics

Resources

License

Stars

Watchers

Forks

Packages

No packages published