PG essay newsletter

Overview

Let’s set up a script that sends one of Paul Graham’s top essays to our inbox every Monday, Wednesday, and Friday. Here’s how we’ll tackle it:

Write the script to fetch a random essay from Paul Graham’s most popular essays
Set up an email to be sent with the essay every Monday/Wednesday/Friday
Configure the schedule so that it “remembers” which essays have already been sent through

Just show me the code

See the finished code on Replit.You can even use the Replit URL to run your job, as long as the Repl is running. Make sure to replace YOUR_EMAIL_ADDRESS with a valid email address and YOUR_RESEND_API_KEY with your Resend API key, and then you can run the following in your terminal:

# This assumes your API key is set in the current env
# BOOPER_API_KEY=sk_...

curl --location --request POST 'https://scheduler.booper.dev/api/jobs' \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $BOOPER_API_KEY" \
--data-raw '{
    "method": "post",
    "url": "https://pg.reichertjalex.repl.co/api/run",
    "body": {
      "recipient": YOUR_EMAIL_ADDRESS,
      "resend_api_key": YOUR_RESEND_API_KEY,
      "exclude": "$state.sent"
    },
    "cron": "0 10 * * MON,WED,FRI"
}'

npx @booper/cli run \
  POST https://pg.reichertjalex.repl.co/api/run \
  --data "recipient=$YOUR_EMAIL_ADDRESS" \
  --data "resend_api_key=$YOUR_RESEND_API_KEY" \
  --data 'exclude=$state.sent' \
  --cron '0 10 * * MON,WED,FRI'

Fetching essay data

We’ll use the cheerio library to parse the essay page HTML. Here’s what that function might look like:

const fetchEssayHtml = async (slug) => {
  const response = await fetch(`http://www.paulgraham.com/${slug}.html`);
  const html = await response.text();
  const $ = cheerio.load(html);
  const essay = $('td[width="435"]');

  return {
    title: $("title").text(),
    text: essay.text(),
    html: essay.html(),
  };
};

Here’s what the output of that might look like:

Full output: https://pg.reichertjalex.repl.co/api/demo
As text: https://pg.reichertjalex.repl.co/api/demo?format=text
As HTML: https://pg.reichertjalex.repl.co/api/demo?format=html

Triggering an email with the essay

First, let’s define a helper function to pick a random essay:

const getRandomEssay = (excluding = []) => {
  // Add as many as you like
  const slugs = ["avg", "say", "nerds", "taste", "gh", "road", "wealth"];
  const options = slugs.filter((s) => !excluding.includes(s));
  const index = Math.floor(Math.random() * options.length);

  return slugs[index];
};

const getRandomEssay = (excluding: string[] = []) => {
  // Add as many as you like
  const slugs = ["avg", "say", "nerds", "taste", "gh", "road", "wealth"];
  const options = slugs.filter((s) => !excluding.includes(s));
  const index = Math.floor(Math.random() * options.length);

  return slugs[index];
};

Now, all we have to do is trigger the email. Here we handle that in a NextJS API endpoint:

export default async function handler(req, res) {
  // Get your free Resend API key at https://resend.com/api-keys
  const resendApiKey = req.body.resend_api_key || process.env.RESEND_API_KEY;
  const recipient = req.body.recipient || "me@example.com";
  const exclude = req.body.exclude || [];
  const essay = req.body.essay || getRandomEssay(exclude);
  const { title, text, markdown, html } = await fetchEssayHtml(essay);
  const payload = {
    from: `pg@resend.dev`,
    to: recipient,
    subject: title,
    text: markdown || text,
    // Wrap the email in a max-width for better readability
    html: `<div style="max-width: 480px">${html}</div>`,
  };
  const resend = new Resend(resendApiKey);
  const email = await resend.sendEmail(payload);

  return res.json({ email });
}

import express from "express";

const app = express();

app.use(express.json());

app.post("/api/pg", async (req, res) => {
  // Get your free Resend API key at https://resend.com/api-keys
  const resendApiKey = req.body.resend_api_key || process.env.RESEND_API_KEY;
  const recipient = req.body.recipient || "me@example.com";
  const exclude = req.body.exclude || [];
  const essay = req.body.essay || getRandomEssay(exclude);
  const { title, text, markdown, html } = await fetchEssayHtml(essay);
  const payload = {
    from: `pg@resend.dev`,
    to: recipient,
    subject: title,
    text: markdown || text,
    // Wrap the email in a max-width for better readability
    html: `<div style="max-width: 480px">${html}</div>`,
  };
  const resend = new Resend(resendApiKey);
  const email = await resend.sendEmail(payload);

  return res.json({ email });
});

app.listen(process.env.PORT || 3000, () => console.log("Server initialized!"));

Keeping track of alert history

When you create a job on a schedule, the scheduler will pass in some metadata into the body of each request. Included in the metadata is the schedule’s state, which can be accessed at req.body.$state and set or updated in the response. Let’s modify the API handler to take advantage of $state:

export default async function handler(req, res) {
  // ...

  const exclude = req.body.$state?.sent || [];
  const essay = req.body.essay || getRandomEssay(exclude);

  // ...

  const email = await resend.sendEmail({...});

  return res.json({ email, $set: { sent: [...exclude, essay] } });
}

We can clean this up a bit by taking advantage of dynamic values in our job schedule configuration. When we set the request body for our job, we can write it like this:

{
  // Pass in `req.body.$state?.sent` as `req.body.exclude`
  "exclude": "$state.sent"
}

If we do this, we can change the line above from this:

const exclude = req.body.$state?.sent || [];

To this:

const exclude = req.body.exclude || [];

Now, assuming you’ve deployed your API endpoint to https://yourdomain.com/api/pg, you can create your scheduled job by running the following script in your terminal with your BOOPER_API_KEY set, and the url modified to the appropriate domain:

# This assumes your API key is set in the current env
# BOOPER_API_KEY=sk_...

curl --location --request POST 'https://scheduler.booper.dev/api/jobs' \
--header "Content-Type: application/json" \
--header "Authorization: Bearer $BOOPER_API_KEY" \
--data-raw '{
    "method": "post",
    "url": "https://yourdomain.com/api/pg",
    "body": {
      "recipient": YOUR_EMAIL_ADDRESS,
      "resend_api_key": YOUR_RESEND_API_KEY,
      "exclude": "$state.sent"
    },
    "cron": "0 10 * * MON,WED,FRI"
}'

npx @booper/cli run \
  POST https://yourdomain.com/api/pg \
  --data "recipient=$YOUR_EMAIL_ADDRESS" \
  --data "resend_api_key=$YOUR_RESEND_API_KEY" \
  --data 'exclude=$state.sent' \
  --cron '0 10 * * MON,WED,FRI'

Documentation

API Reference

Jobs

Schedules

Logs

Miscellaneous

PG essay newsletter

Overview

Fetching essay data

Triggering an email with the essay

Keeping track of alert history

​Overview

​Fetching essay data

​Triggering an email with the essay

​Keeping track of alert history

Overview

Fetching essay data

Triggering an email with the essay

Keeping track of alert history