Easily Build Static HTML Sites from JSON with AWS Lambda and TiddlyWiki

Generating static HTML websites has quickly gone from the obscurity of Jamstack geekdom to the Notion-obsessed mainstream. If you are like me and you believe low-code is the warm porridge that will feed the future internet, you have probably searched for something right in the middle of those two extremes: an easy way to generate static HTML from JSON using AWS Lambda, but without actually having to write any javascript.

In this tutorial, I’ll show you how to skip most of the extra work and nuances of traditional Static Site Generators like Gatsby (the cold porridge). And while there’s nothing wrong with hot porridge, at Digital Mark we like to taste the grit of our concoctions so we can add spice accordingly — and sometimes the trendy Notion-based Static Site Generators like Super and Potion can obscure a little too much fun and flexibility for us.

If Goldilocks liked computers as much as she likes porridge, she’d solve this problem by using TiddlyWiki. So that’s what we’re gonna use.

Table of Contents

1. Intro to TiddlyWiki as a low-code Static Site Generator.
2. Choosing and splitting an HTML template.
3. Creating your Lambda function with TiddlyWiki and no code.
4. Generating your first static page with JSON in the Lambda console.
5. Next up — Generating HTML pages with your Lambda from external JSON (like Gsheets or Airtable).

---

Low-code Static Site Generator using TiddlyWiki

Hugo, Gatsby, Ricardo… they’re great frameworks for getting new users from Zero to Hello World. I even made up the last one and it still sounds approachable enough.

But once you take the template out of the box, you will inevitably need to customize some things. What if I want to generate a list page? Or what if I want to integrate some jquery I found on Stackoverflow into my pages?

There’s a whole subset of geeks out there who can’t take the time right now to learn a bunch of opinionated javascript, just to render some filtered list in JSX and make a custom page. It’s too easy to get stuck with this stuff. This is bad because it turns people away from web dev, which means it turns people away from creating more cool things online for us all to enjoy.

Well, what if I told you there is an easy low-code tool to generate static pages with AWS Lambda?

TIDDLY THE WIKI

If you haven’t heard of TiddlyWiki, you may think I am playing the made-up name game again. But TiddlyWiki’s our favorite low-code framework. It’s been around for something like 20 years. It’s solid. And it’s easy.

For the purpose of generating static HTML, you can think of TiddlyWiki as just a super-simple IDE for setting up your custom pages. High-level, all you have to do is:

• Open a new TiddlyWiki instance in your browser.
• Copy-paste your HTML template into your Wiki.
• Add some {{ curly brackets }} for data that will be passed in from your JSON inputs.

Once you do that, you only need one command to generate static HTML based on your template. The beauty of this whole process is that TiddlyWiki can easily export itself as a Lambda function, so you can run that command in AWS on-demand. That means you can start sending data into to your Wiki and generating HTML pages in S3 at will. All without coding any javascript yourself!

Using this method, you can generate static HTML pages from a Gsheet, or from boards in Airtable or Trello. Or you can use any other way of creating and sending JSON, such as from another TiddlyWiki instance serving as your custom CMS. Mind blown!

Choosing and splitting static HTML for the Lambda

THE TASK

For this exercise, my challenge is to create a simple HTML page to display a list, and update the list daily via my Lambda function. The list should be sorted alphabetically, and contain all of the topics covered on the news website Purple News.

Purple News was born out of a client need to track the way several news sources were covering the topic of “Voting” during the 2020 US presidential election. Using AWS Athena to scale up SQL query automation, we extended this concept to track lots of different topics — and used TiddlyWiki-on-Lambda to generate every page on the site!

Each day on Purple News, hot topics in the news are automatically queried and collated for easy reading in an HTML template. The entire HTML template is generated by TiddlyWiki. The idea is to help readers more objectively identify and compare the political bias which is present in all reporting.

Whenever a new topic/keyword like “Travel” is created, a small JSON payload about the topic is sent to API Gateway. This triggers the Lambda function to “print” the HTML page to S3. The point of all this is that the code you need for the Lambda is easily generated with TiddlyWiki, with zero need to know how to code javascript.

So far on Purple News, we’ve already got a handy Lambda function set up to generate a page for each individual topic; but what if we want to build an index page, like a sitemap, so users can browse through all of the topics covered on the site?

THE HTML

In order for a visitor to see a sitemap of every topic on Purple News, I’ll want to utilize an HTML format that will allow me to easily append new Topics to the list each day via my Lambda. Nothing too fancy here, just a bulleted list will do:

<ul id="purpleTopics">
  <li><a href="/arizona">Arizona</a></li>

  <li><a href="/belarus">Belarus</a></li>

  <li><a href="/china">China</a></li>
</ul>

Since my basic JSON payload will include the proper name of the topic, along with the relative link to the page, my curly “mustache” template for list items will look something like:

<li><a href={{topic_link}}>{{topic_name}}</a></li>

It would also be really cool if I could allow users to search the list right there on the page. This would be great since the list is growing, and it would save me the trouble of setting up site-wide search via another Lambda.

I found this simple boilerplate code on w3schools that will work perfectly. It even gives me some nice CSS styling for my list. This should be quick!

The page will consist of:

• Basic HTML opening headers.
• CSS styles from w3schools to style the list items.
• HTML body with my {{ curlies }}, which my TiddlyWiki-on-Lambda will turn into a list of news topics.
• JS snippet from w3schools for the search feature.

THE STEPS

1. Install the Node.js version of TiddlyWiki on your local machine.

The only prerequisites on your machine are Node.js and NPM, the Node Package Manager.

sudo npm install -g tiddlywiki

To verify that worked, make sure you get a response when you check your TiddlyWiki version:

tiddlywiki --version

2. Start a new Wiki called TopicBrowser.

Let’s start by creating a new parent directory for our little project today called TopicWikis.

mkdir TopicWikis
cd TopicWikis

Ultimately we’ll be making two Wikis in this directory: one for our local dev environment, and another that we’ll use specifically for generating our Lambda function.

Let’s go ahead and create our first Wiki.

From the TopicWikis directory, we’re going to initiate a new Wiki to set up our HTML template on our local machine. We’ll call the Wiki “local_Wiki.” To make the Wiki easy for us to edit right in the web browser, we’ll pull in TiddlyWiki’s prebuilt “server” components, which will run a simple webserver on our machine and serve our Wiki.

tiddlywiki local_Wiki --init server

You’ll see that the tiddlywiki command created a new directory called local_Wiki.

We are now ready to run TiddlyWiki on Node.js on our local machine and interact with our Wiki. Let’s launch our local_Wiki so we can paste-in our HTML from w3schools.

tiddlywiki local_Wiki --listen port="8090"

This will launch your new Wiki locally on port 8090. The response in the terminal will look as follows (note the local url at http://127.0.0.1:8090). You should leave this terminal pane alone while it’s running.

/*EXPECTED RESPONSE FROM TERMINAL*/

syncer-server-filesystem: Dispatching 'save' task: $:/StoryList 
Serving on http://127.0.0.1:8090
(press ctrl-C to exit)

We’re heating up! Now you can go ahead and visit port 8090 in your browser and see your new Wiki.

3. Paste your HTML into the Wiki as a Tiddler called TopicPage.

You’ll see under the “My TiddlyWiki” page title in the sidebar that there’s a handy plus sign (+). Go ahead and click here to create a new draft “Tiddler.”

Don’t get hung up on the terminology here — a Tiddler is just like a “Post” on WordPress or any other platform.

Inside of our draft Tiddler, let’s borrow all of the code for the page from w3schools and paste it in. There’s one important trick here: paste the code inside of two backticks. The backtick is usually next to the number 1 on your keyboard.

Here’s how the full code from w3schools will look at this point inside of my TopicPage Tiddler, including basic HTML tags. I’ve commented out the searchicon.png for the search box, since we want everything contained in one page right now, not referencing outside files. Note the backticks at the top and bottom of the code.

`
<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
* {
  box-sizing: border-box;
}

#myInput {
  /*background-image: url('/css/searchicon.png');*/
  background-position: 10px 12px;
  background-repeat: no-repeat;
  width: 100%;
  font-size: 16px;
  padding: 12px 20px 12px 40px;
  border: 1px solid #ddd;
  margin-bottom: 12px;
}

#myUL {
  list-style-type: none;
  padding: 0;
  margin: 0;
}

#myUL li a {
  border: 1px solid #ddd;
  margin-top: -1px; /* Prevent double borders */
  background-color: #f6f6f6;
  padding: 12px;
  text-decoration: none;
  font-size: 18px;
  color: black;
  display: block
}

#myUL li a:hover:not(.header) {
  background-color: #eee;
}
</style>
</head>
<body>

<h2>My Phonebook</h2>

<input type="text" id="myInput" onkeyup="myFunction()" placeholder="Search for names.." title="Type in a name">

<ul id="myUL">
  <li><a href="#">Adele</a></li>
  <li><a href="#">Agnes</a></li>

  <li><a href="#">Billy</a></li>
  <li><a href="#">Bob</a></li>

  <li><a href="#">Calvin</a></li>
  <li><a href="#">Christina</a></li>
  <li><a href="#">Cindy</a></li>
</ul>

<script>
function myFunction() {
    var input, filter, ul, li, a, i, txtValue;
    input = document.getElementById("myInput");
    filter = input.value.toUpperCase();
    ul = document.getElementById("myUL");
    li = ul.getElementsByTagName("li");
    for (i = 0; i < li.length; i++) {
        a = li[i].getElementsByTagName("a")[0];
        txtValue = a.textContent || a.innerText;
        if (txtValue.toUpperCase().indexOf(filter) > -1) {
            li[i].style.display = "";
        } else {
            li[i].style.display = "none";
        }
    }
}
</script>

</body>
</html>`

Once your code is pasted in, don’t forget to click the check mark in the top right of the draft Tiddler to save your work!

4. Test your work so far by generating the w3schools sample page locally.

In a new terminal window, navigate to your TopicWikis directory.

cd TopicWikis

Now generate the w3schools example HTML page by running:

tiddlywiki local_Wiki --render [[TopicPage]] [is[tiddler]addsuffix[.html]] "text/plain" 

If all has gone well up to here, your directory structure under TopicWikis will look like below, with an “output” directory newly created by your –render command.

You can now open up TopicPage.html and it should look just like the example from w3schools!

5. Replace the sample list of items with our {{ curlies }}.

Now it’s finally time to get the heart of the matter, and templatize our HTML so it can receive JSON inputs.

Remember our curly bracket, aka “mustache” template, from earlier? Let’s keep it handy:

<li><a href={{topic_link}}>{{topic_name}}</a></li>

Now, back in our local_Wiki in our web browser, open up the TopicPage Tiddler by clicking the pencil icon. Now we’re back to our raw HTML from w3schools.

In the center of our code, we have the HTML bullets for our list. We note that the opening and closing ul tags won’t ever change. It’s important to note again here that any elements that won’t change based on our JSON inputs should be enclosed in backticks. We’ve got four (4) backticks in total at this point.

`/*all of the opening HTML from earlier goes here*/

<ul id="myUL">
`

<li><a href={{topic_link}}>{{topic_name}}</a></li>
`
</ul>

/*all of the closing HTML from earlier goes here*/

`

There’s only one problem here. Mustache templates are all well and handsome, but how do we instruct TiddlyWiki to generate a list of all the topics, and display them in alphabetical order using our mustache template?

Well, thanks to the power of TiddlyWiki’s low-code WikiText syntax, it’s suuper easy to generate a filtered list and sort the items we pass in from JSON.

TIDDLYWIKI LIST FILTER SYNTAX

A key concept of TiddlyWiki is that “everything is a Tiddler.” We have already created one Tiddler: the TopicPage. Next, we should note that the actual topics passed into our Wiki later will also become Tiddlers themselves.

Tiddlers are analogous to Posts in the frontend, and analogous to javascipt objects under the hood. They can contain any fields and content you want. For example, our TopicPage Tiddler contains the following fields:

• Title field: TopicPage
• Text field: My HTML code enclosed in backticks

Likewise, our actual topics will be passed into our Wiki as Tiddlers, and will contain the following fields:

• Title field: proper topic_name of the topic, like “Joe Biden”
• Link field: href topic_link for the topic, like “/joebiden/”
• Tag field: we’ll tag each topic as “Topic,” just to make it easy to filter these Tiddlers out later

By using the Tag field to include only those Tiddlers in our Wiki which have the tag “Topic,” WikiText allows us to generate a list of all topics by simply enclosing our curlies in a couple of special WikiText list tags. Note the addition of the double exclamation point in front of the field name, which is how field references work in WikiText.

<$list filter="[tag[Topic]]">
<li><a href={{!!topic_link}}>{{!!title}}</a></li>
</$list>

That’s all you need to generate a bullet list of every topic you create. It’s this magic right here, the speed and simplicity of the list filter, which makes TiddlyWiki such a powerful low-code tool. Want to play with the list a little? Easy!

/*sort alphabetically*/
<$list filter="[tag[Topic]sort[title]]"> </$list>

/*list only those topics modified in the past week*/
<$list filter="[tag[Topic]days[-7]sort[title]]"> </$list>

Using WikiText, we can extend the filtering capability to create all types of conditional data displays and formatting. Combine this with the ability to create WikiText macros (or actual js macros), and you’ve got yourself a low-code tool for the ages.

const myPromise = new Promise((from_me, to_you) => {
  WikiText = {
    way freakin' easier than this.('foo');
  }, 100);
});
});)))});)) fml ))});

To test our progress locally, let’s go to the plus sign in the side bar again and create two sample topics of our own. I’ll do one for the current President and one for the current Vice President.

Note how I added two fields in addition to the Title:

• the word “Topic” in the tag field, and
• the href link to /joebiden in a new field I added called “topic_link.”

Make sure you always click the check mark in the top right to save each Tiddler when you’re done editing it.

Now that we have two topics loaded in as examples to test our work, let’s generate our HTML file once more locally, and see if our list filter did the trick. Our final HTML in the TopicPage Tiddler should look like below.

Note two small edits:

• I changed the “My Phonebook” header from the w3schools template to “Topics,”
• I added six (6) additional backticks to the list widget, in order to escape all of the html code that’s inside of the mustache template. Remember: everything that’s not your JSON variables should be enclosed in backticks.

`

<!DOCTYPE html>
<html>
<head>
<meta name="viewport" content="width=device-width, initial-scale=1">
<style>
* {
  box-sizing: border-box;
}

#myInput {
  /*background-image: url('/css/searchicon.png');*/
  background-position: 10px 12px;
  background-repeat: no-repeat;
  width: 100%;
  font-size: 16px;
  padding: 12px 20px 12px 40px;
  border: 1px solid #ddd;
  margin-bottom: 12px;
}

#myUL {
  list-style-type: none;
  padding: 0;
  margin: 0;
}

#myUL li a {
  border: 1px solid #ddd;
  margin-top: -1px; /* Prevent double borders */
  background-color: #f6f6f6;
  padding: 12px;
  text-decoration: none;
  font-size: 18px;
  color: black;
  display: block
}

#myUL li a:hover:not(.header) {
  background-color: #eee;
}
</style>
</head>
<body>

<h2>Topics</h2>

<input type="text" id="myInput" onkeyup="myFunction()" placeholder="Search for names.." title="Type in a name">

<ul id="myUL">
`

<$list filter="[tag[Topic]]">
`<li><a href=`{{!!topic_link}}`>`{{!!title}}`</a></li>`
</$list>


`
</ul>

<script>
function myFunction() {
    var input, filter, ul, li, a, i, txtValue;
    input = document.getElementById("myInput");
    filter = input.value.toUpperCase();
    ul = document.getElementById("myUL");
    li = ul.getElementsByTagName("li");
    for (i = 0; i < li.length; i++) {
        a = li[i].getElementsByTagName("a")[0];
        txtValue = a.textContent || a.innerText;
        if (txtValue.toUpperCase().indexOf(filter) > -1) {
            li[i].style.display = "";
        } else {
            li[i].style.display = "none";
        }
    }
}
</script>

</body>
</html>
`

We can pre-verify that we’re on the right track just by saving our TopicPage Tiddler in the browser, and noticing how TiddlyWiki escapes all of our static HTML with red font (that’s the stuff enclosed in backticks), and renders our list of Topic {{ fields }} with plain text as intended.

But to really be sure we’re still on track, let’s generate the TopicPage.html file again locally and double check. Back in the terminal:

tiddlywiki TopicBrowser --render [[TopicPage]] [is[tiddler]addsuffix[.html]] "text/plain" 

You’ve now overwritten the TopicPage.html you created earlier. If all went well, hard-coded Calvin and Billy Bob have gone the way of the LaserDisc, in favor of our fancy robot-created list of topics.

Creating your Lambda function with TiddlyWiki

We are now ready to generate our Lambda function so that we can upload it to AWS and start using it in the cloud.

You can do this all in the command line with the AWS SDK, but the steps below allow you to skip local setup of the SDK, and upload the Lambda directly to the AWS Console in your browser.

When you upload TiddlyWiki to AWS as a Lambda function, all of the dependencies you need will be packaged into an index.js file. We will be generating this file on our local machine using TiddlyWiki. To upload the function to AWS, we’ll need to zip the index.js file with a package-lock.json file taken from a fresh install of TiddlyWiki itself.

Let’s generate our index.js and package-lock.json files.

1. Install a clean TiddlyWiki node package in a new directory.

From your TopicWikis parent directory, create a new directory for generating our Lambda function.

mkdir lambda_Wiki

Our TopicWikis directory for our project today should now look like this.

Next, we want to install TiddlyWiki anew in this folder, even though we already have TiddlyWiki installed globally on our machine. We do this using the prefix flag in NPM. If you’re like me and you mess up directory locations a lot, it’s better to start in your home directory, and enter the entire path of your lambda_Wiki directory.

Note also that I’ll be specifying v5.1.23 of TiddlyWiki. This process has been tested with 5.22 and 5.23, but not any other versions.

cd

npm install --prefix ./your_parent_dirs/TopicWikis/lambda_Wiki tiddlywiki@5.1.23

Now that we have a local sandbox for TiddlyWiki in this folder, let’s create our first Lambda function inside the lambda_Wiki directory.

Navigate back to the lambda_Wiki folder.

cd TopicWikis/lambda_Wiki

Next, initiate a new instance of TiddlyWiki, but use the “aws” edition, instead of the “server” edition we used in our first Wiki instance.

tiddlywiki TopicLambda --init aws

This created the TopicLambda directory, a new tiddlywiki.info file and a tiddlers folder inside of the TopicLambda directory. We have now created two instances of TiddlyWiki on our machine: local_Wiki and lambda_Wiki/TopicLambda. Now we’re almost there!

2. Paste over your Tiddlers from the local_Wiki to the lambda_Wiki/TopicLambda.

Now, we’re pretty much ready to output our Lambda function so we can upload it to AWS. We just need to make sure our Lambda function has our cool HTML template in it!

We’ll go ahead and just copy-paste our Tiddler files from our local_Wiki to our lambda_Wiki. We’ll even throw in Joe Biden and Kamala Harris for good measure. You’re overwriting the entire tiddlers directory in lambda_Wiki/TopicLambda in this step.

3. Generate your Lambda function with one terminal command.

Now we’re ready to generate our Lambda!

From the lambda_Wiki directory:

tiddlywiki TopicLambda --rendertiddler $:/plugins/tiddlywiki/aws/lambdas/main index.js text/plain

Now look into lambda_Wiki/TopicLambda/output and find the index.js file. That’s your Lambda function!

4. Zip and upload your Lambda function to AWS.

You may have had to jump through a hoop or two, but you just created a super-powerful Lambda function without writing a single line of javascript. Good on ya!

Now, all you have to do to release your HTML robot into the wild is create a zip archive with your index.js file from the last step, and your package-lock.json at lambda_Wiki/package-lock.json.

Then login to the AWS console and create a new Lambda function in Node.js (visit Lambda and click “Create function”).

Note that I have named my function tw-topicbrowser and selected the latest supported version of Node.js for my runtime. One trick here: your Lambda will not work unless you give the function permission to write to S3. It doesn’t take long to set this up, check out this link in AWS docs.

Once you have an Execution Role with appropriate permissions, go ahead and upload your zip file holding your index.js and package-lock.json files.

Now we’re ready to generate our HTML page into S3 using our Lambda!

Generating static HTML page with Lambda (console)

Now that you have a function up in the cloud, you can send JSON into AWS from anywhere and generate HTML pages in S3 from Lambda.

To make sure we structure incoming JSON correctly, let’s practice by generating a page right from the Lambda console using the Test feature.

1. Format your JSON.

First, let’s prepare the JSON that we want to pass in.

From the Github repository for TiddlyWiki, we can see how to properly structure our command in AWS.

The format is like this:

{
	"commands": [
		"--aws","s3-rendertiddler","HelloThere","eu-west-2","my-bucket-name","rendered.html"
	],
	"tiddlers": [
		{
			"title": "HelloThere",
			"text": "Hello from {{Platform}}."
		},
		{
			"title": "Platform",
			"text": "TiddlyWiki"
		}
	]
}

Following this format, we’ll note that there are other parameters we can add to the s3-rendertiddler command. When generating static HTML in S3, I have found it useful to use the render type of “text/plain” and later specify the MIME type of the saved file as “text/html.” I’ve left placeholders in my command for two other command flags we won’t use today.

Finally, you’ll note that I pass in two additional Topic Tiddlers for this Lambda trigger, Arizona and Belarus.

{
  "commands": [
    "--aws",
    "s3-rendertiddler",
    "TopicPage",
    "us-east-1",
    "purplenewstestbucket",
    "TopicPage.html",
    "text/plain",
    "",
    "",
    "text/html"
  ],
  "tiddlers": [
    {
      "title": "Arizona",
      "topic_link": "/arizona",
      "tags": "Topic"
    },
  {
      "title": "Belarus",
      "topic_link": "/belarus",
      "tags": "Topic"
    }
  ]
}

Go ahead and paste this JSON into a new Test configuration in the Lambda console.

Click the arrow next to the Test button and select “Configure test event.”

And paste in the entire JSON test event from above.

2. Run your Lambda and find your HTML file in S3.

Before I run my Lambda, I’m going to complete two little tricks of the TiddlyWiki-on-Lambda trade:

A) Juice up my Lambda by increasing the allocated memory and the max execution time. 512MB and 30 seconds is usually a good fit for a smaller Wiki like this one.

B) Add an environment variable to help the function identify our payload. To accommodate a small quirk in the way payloads get passed through our workflow and into TiddlyWiki, here I’m simply telling TiddlyWiki to look for its command in the event.body portion of the payload which ultimately reaches the function.

DRUMROLL PLEASE….

Navigate back to the “Code” tab of your Lambda, select the test event you just created, and click Test!

If you’ve followed along, first of all, congratulations for your excellent attention span. Also, your wicked cool HTML page has been dropped into S3 and is ready for viewing.

To see your file, the culmination of your low-code work, you can just download TopicPage.html to check it out. Nice!

You can also see a live version of this template being used on the Purple News topics page.

If you encountered an error, don’t give up now! The most common error will be a permissions misconfiguration. You’ll get an error about not being able to write to the bucket. Double check that the “Execution Role” in the “Permissions” tab has permission to write to this bucket. Otherwise, hit me on twitter or something, you’ve made it this far!

Generating static HTML with JSON, API Gateway, and Lambda

Now that we know our function works, we’ll want to go about getting the true benefits of Lambda. By setting up Triggers for our function in API Gateway, we can generate static html pages in S3 from JSON at will.

In my next tutorial, I will be going over some of the cool ways we can create and send JSON into the Lambda function we created today. You can use tools like Google Sheets to generate static HTML in Lambda, for example. You can even set up another TiddlyWiki on your machine and configure it to post topics to your Lambda — meaning you’ll have a fully-customizable backend CMS on your hands!

By setting up an endpoint in API Gateway, I will be using Body Mapping Templates to ingest JSON from any source, and transform it into TiddlyWiki commands and tiddlers.

Here’s a sneak peek at a body mapping template. Low-code may not mean no-code, but it’s easier than it looks!

## Set the variable inputRoot to the root JSON document
#set($inputRoot = $input.path('$')) 
{
  "commands": [
    "--aws",
    "s3-rendertiddler",
    "TopicPage",
    "us-east-1",
    "purplenewstestbucket",
    "TopicPage.html",
    "text/plain",
    "",
    "",
    "text/html"
  ],
  "tiddlers": [
    {
      "title": "$inputRoot.title",
      "topic_link": "$inputRoot.topic_link"

    }
  ]
}

---

If you liked this post, follow me on Twitter @philwonski so you know when the follow-up comes out. There’s also lots of great (and less esoteric) content being produced by our team on a regular basis — be sure to find us on LinkedIn, and sign up to our newsletter in the footer of this page.