Cloud on Clément Joly – Open-Source, Rust & SQLitehttps://cj.rs/tags/cloud/Recent content in Cloud on Clément Joly – Open-Source, Rust & SQLiteHugo -- 0.154.3enClément JolyMon, 09 Mar 2026 22:59:21 +0000Link Aggregator Infrastructurehttps://cj.rs/blog/link-aggregator-infrastructure/Thu, 21 Mar 2024 14:40:33 +0000https://cj.rs/blog/link-aggregator-infrastructure/A look at the surprisingly simple infrastructure of link aggregators like HackerNews or Lobste.rs.

⚠️ DISCLAIMER

This is not a comment on Reddit’s future profitability or any reflection on the stock performance. This is a technical discussion on how a relatively simple link-aggreggator can be hosted, with the trade-offs.

Reddit went public today. Its IPO document states1 that they use AWS and GCP. But I cannot find a break-down of infrastructure costs separately.

That makes me wonder: what does the infrastructure costs of a pure link-aggreggator look like?

I’ll go over two sites aggregating links related to computing, Lobsters and Hacker News. They are relatively high-traffic websites. Sites popular on Hacker News in particular often go down due to the sudden popularity2. It’s a little less common with Lobsters, but it happens there as well.

Lobste.rs

Lobsters is very transparent on its infrastructure:

Lobsters is hosted on three VPSs at DigitalOcean: a s-4vcpu-8gb for the web server, a s-2vcpu-4gb for the mariadb server, and a s-1vcpu-1gb for the IRC bot […] we use restic for backups to b2

https://lobste.rs/about

Using public prices, those servers cost $78 ($48+$24+$6) a month to run. Obviously, there are other costs for monitoring, backups, managed DNS3…

I’ve requested some numbers on the load the site is facing and infrastructure utilization. I’ll update that blog post with the results.

Hacker News

It’s harder to find details on Hacker News’s infrastructure, but the moderator of the site answered questions about the infra in this thread4:

We’re recently running two machines (master and standby) at M5 Hosting. All of HN runs on a single box, nothing exotic:

  CPU: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz (3500.07-MHz K8-class CPU)
  FreeBSD/SMP: 2 package(s) x 4 core(s) x 2 hardware threads
  Mirrored SSDs for data, mirrored magnetic for logs (UFS)
https://news.ycombinator.com/item?id=16076041

Number of daily requests has gone up closer to 6M

https://news.ycombinator.com/item?id=28479595

We use an Nginx front end for that [caching]. It all runs on the same box though.

https://news.ycombinator.com/item?id=28496642

So a CPU from 2016 handles the load for a very popular site.

No CDN either, requests are made to news.ycombinator, with DNS records pointing to M5 Hosting IPs (2606:7100:1:67::26 and 209.216.230.207)5.

Conclusions

Those two popular link-aggregator offer to submit and comment on links, two historical features of Reddit. Of course now Reddit also has chat, image hosting and presumably a bunch of other features. It also has “73.1 million daily active uniques (“DAUq”), around the world”, probably orders of magnitude more than Hacker News. Finally, Reddit might6 theoretically have higher uptime, because Hacker News and Lobsters sometimes have their single-point-of-failure fail, for instance during an upgrade.

After those incidents though, users tend to post understanding comments, pointing out that they thought their Internet connection was faulty, before thinking that the aggregator could be down. This sort of simple hosting might be the right trade-off for a non-profit link-aggregator: simple, so rarely down due to an operational mistake and with base components simple enough that users are understanding when an outage happens.

  1. “Currently our cloud service infrastructure is run on our cloud services providers (“CSPs”), which are currently Amazon Web Services and Google Cloud Platform”. ↩︎

  2. And those links are only from what I could quickly find in comments from the last two month. ↩︎

  3. They don’t mention using a CDN on Lobsters. It’s consistent with what I observe loading the homepage, where all requests are made to the host lobest.rs. A quick DNS lookup returns 67.205.189.7 and 2604:a880:400:d0::2082:1001, and both IPs are owned by DigitalOcean5. ↩︎

  4. These details are consistent with this comment from 2023. ↩︎

  5. According to ipinfo. ↩︎ ↩︎

  6. It’s hard to find reliable uptime data for the 3 sites and compare. ↩︎

]]>