Build a WooCommerce Dashboard with SMS using Twilio, Pipedream and TiddlyWiki

WooCommerce is great for getting a store up and running quickly and avoiding fees. But some small business owners may find the WordPress backend a bit cumbersome. It’s busy back there. What if we could create a WooCommerce Dashboard that strips out all the bells and whistles and focuses on Orders? In this tutorial I’ll do just that. Then I’ll integrate the dashboard with customer SMS, using Twilio, Pipedream and TiddlyWiki.

Table of Contents

1. The Problem – Using the WooCommerce backend to process incoming Orders.
2. The Solution – Build a Woo Dashboard with TiddlyWiki (free download).
3. The Magic – Processing Woo Orders using Pipedream.
4. The Bliss – Integrating Twilio SMS to contact customers about Order status.

The Problem – Using the WooCommerce backend

If you’ve ever set up a store with Woo for a small business customer, you know this pain. You probably spent as much time training the customer on the backend as you did building the actual store.

Just to view an Order and begin processing, you’ve got to:

1. Find the right place on a busy left pane.
2. Load a long list of Orders, which can take time.
3. Find the Order you’re looking for in the list, then wait for it to load.
4. Scroll down past a bunch of pesky messages, only to reach the intimidating screen above.

Also, the list of Orders may not refresh as quickly as you’d like (or at all). So you may not know a new Order came in unless you are simultaneously monitoring an email account.

In fact, email is a popular way that Woo users avoid this mess. You can set up a dedicated email account to receive Orders, and that will solve some of these problems. Woo will send over a neat summary of your new Order, and you can use email folders to keep track of Order status from there.

This email isn’t terrible, but it still leaves some gaps. What if I want a button to update the status of the Order without navigating the Woo backend? And what about quickly sending the customer text messages to update them on Order status?

Email is probably fine for a low-volume store, but not for a store that gets dozens of Orders per day. A busy store — like a food business — will need a slick dashboard. The dashboard should strip out info and links we don’t need, and focus on the important Order info. It should give us an easy way to view and organize Orders. And it should be extensible, so we can communicate with Woo and with the customers.

The Solution – Build a WooCommerce dashboard with TiddlyWiki

Enter TiddlyWiki, our favorite low-code development tool. TiddlyWiki will allow us to simplify the display and management of Orders as they come in. By using simple HTTP requests in the browser, we’ll also be able to POST Order updates to the net, right from the dashboard: this will allow us to text message our customers, update Order status in Woo, and whatever else you can imagine.

This Little Wiki had Roast Beef

If you’ve ever been to New York / New Jersey, you’ll know that we are serious about our deli meats in these parts. Whether it’s Taylor Ham, Hot Pastrami, or just some rare Roast Beef cooked right in the store, we take pride in a high-quality sandwich.

My family owns and operates the Town Hall Deli in South Orange, NJ. It’s a local institution. On any given day, as many as a couple hundred folks will come in for the highest quality ingredients and unique recipes. And the signature sandwich, the Sloppy Joe, isn’t the messy pile of ground beef you had in grade school: it’s a local delicacy that takes 10-15 minutes to prepare, meaning it’s also the perfect thing to order ahead online before you drop by.

A Peek Behind the Deli Counter

In any food business, Order accuracy can be a big challenge. Was it Mrs. Pearson who likes her whole wheat extra toasted? Or was she the one who requested the gluten-free bread?

Online ordering can solve a lot of problems associated with the literal game of telephone. But it only really works if reviewing Orders is dead simple. Toward that end, I created a clean dashboard so the deli staff can view incoming Orders on a tablet in the store.

When new Orders come in, they’re colored green, and the time and other basic details are clearly displayed. By pressing an Order, the staff gets an opportunity to review the full details, and send out a text message to let the customer know what’s going on. Once they engage the customer, the Order will change colors in the home screen (blue means the Order “Ready Time” was confirmed, and the food is being prepared).

The beauty of using TiddlyWiki for this task is its low-code WikiText syntax. With WikiText, this dashboard was created in hours, not days or weeks. And it’s also easy to come back and add more features and custom text messages.

To set up the “DeliDash” dashboard and customize it yourself, you can download it here on Github. Node js is the only prerequisite to run it, although I add pm2 and rclone to the mix in later steps. I like to set these up on a Digital Ocean VPS: you can follow my guide from another post to set up the VPS and secure it with a login screen using Cloudflare Argo Tunnels (just replace the Smithy POS app in the tutorial with this DeliDash repo). Or you can run it however else you like to run Node apps, including on a Raspberry Pi on a local network.

The Magic – Processing WooCommerce Orders using Pipedream

TiddlyWiki helped us ship something quickly, and tweak the dashboard UI based on feedback from the clerks in the deli. But how do we actually get the data out of WooCommerce and into our dashboard in the first place?

Pipedream is the hero here. If you haven’t heard of Pipedream, it’s a tool for quickly integrating SaaS APIs, and for building multi-step workflows with data from anywhere. You can compare Pipedream to Zapier — but if you’re a developer, they’re hardly comparable, because Pipedream easily takes the cake. Pipedream has a developer-focused UI, which makes it crazy fast to build data workflows.

In the old days before API tools like Pipedream existed, I’d probably use a simple Express server to catch JSON data from WooCommerce like this:

There’s certainly nothing wrong with this approach. In fact, I began this way, and even set up a basic Express server for catching Orders (it’s available on Github in the express_test directory). So why add anything else to the mix?

Challenge 1: Navigating WooCommerce JSON data

After I set up a basic Express server to catch WooCommerce webhooks, I went ahead and completed a test Order to see what the payload looked like. It wasn’t pretty.

Parsing this JSON on the server into a pretty user display can be a tedious task, and time is limited. The way WooCommerce webhooks presents Orders in JSON creates a couple of significant time-related issues:

1. I need a quick way to get the JSON paths and pull out the data that I need from this mess of a JSON payload.
2. I don’t want to have to write a lot of code to parse all of this information and display it neatly in the dashboard. Ideally, I’d like to just pull out a couple of important things like Ready Time and Phone Number. If I were to try and build a display for this full payload on the server, I’d have to code some logic to loop through various kinds of Orders/items/add-ons. I’d basically be recreating the neat little WooCommerce email that goes out, which seems crazy.

Luckily, Pipedream can help me resolve both of these issues super quickly:

1. First of all, ugly JSON is no match for Pipedream. I’ll be able to identify the keys and values I need — and pass them through a bunch of steps — in no time.
2. Second, since Pipedream supports multi-step “Workflows,” why recreate my own display for the actual Order at all? With Pipedream, I can just create a Workflow that will carry the nice-looking email from WooCommerce through each step, and talk to the WooCommerce API to get any info I’m missing.

To make use of the ready-to-go Order email from Woo, the first side of the Pipedream workflow ends up looking like this:

Here’s a quick explanation of what I’m doing here:

1. WooCommerce generates a handy New Order email with every order. I am using that to trigger my Workflow.
2. By pulling the Order Number from the email subject, I can now ping the Woo API to Tell me more about the Order. This will help me request the particulars I need in my dashboard, such as Ready Time. Ready Time comes from Woo’s “Custom Checkout” fields, which aren’t included in the email by default.
3. Woo will send back all the Order Deets, and I can now easily process those in the Pipedream GUI. Importantly, I am also using the HTML from the original Woo email, and passing it through the whole Workflow. I’ll use this HTML in the dashboard, so I don’t have to build a new display for the Order from scratch. Huge time saver.

For larger systems, a variation here might be to edit the Order email template in WooCommerce to include all the data we need. This of course saves resources on the callback to Woo. The only problem with this approach is that the store is just a v1, and much could change in the checkout form and experience in Woo. By polling the API, I can predictably pick up what I need, and won’t have to constantly update the content of the email with PHP.

Here’s how to get this Workflow going in Pipedream:

1. Get a WooCommerce API Key in your Woo backend.

First, navigate to your WordPress+Woo backend. Under WooCommerce in the left pane, click “Settings.”

Then create a new API key in the Advanced tab by clicking “Add key” under “REST API.”

2. Add your Woo store as one of your “Accounts” in Pipedream.

Once your Woo API key is created, you’ll need to add the key as one of your “Accounts” in Pipedream.

After signing up for Pipedream, visit “Accounts” in the left pane and click “Connect an App” on the right.

Then, in the dialogue, search for WooCommerce and click. Now you’ll be able to enter in your WooCommerce API key.

3. Prepare an endpoint for receiving WooCommerce Order emails.

Next, visit Workflows and create a New workflow.

Next, name your workflow and select “Email” on the right.

You’ll now see that Pipedream provides an email address for receiving WooCommerce emails and triggering the rest of our Workflow. Copy that email and paste it in your WooCommerce >> Settings >> Emails tab.

Now you’re ready to place your first test Order and see what happens!

Challenge 2: Monitoring and error handling

Now’s a good time to highlight one of the other reasons Pipedream is a real hero. If I set up an Express server and do just the basic Woo –> Express –> TiddlyWiki workflow, it may seem simple in a diagram; but in reality, I’d waste a lot of dev time on troubleshooting. I’d have to set up a way to log and analyze errors, for example. Later, I’d also want a way to notify myself of any errors, which is problematic because I don’t let email anywhere near the server my dashboard is running on.

Wouldn’t it be nice if there was a single place for editing my code, viewing incoming messages, and watching my event play out?

Thanks Pipedream. Sandwiches on me.

4. Send in a test Order.

By now you are anxious to see one of these multi-step API workflows in action. Here we go — let’s complete a fake Order in our store, now that we have entered Pipedream’s email dropbox into Woo.

We can see that the Email subject line is available with the Order number right there for us. We can also see that we get a nice pre-crafted string of HTML which we can use to display the Order.

5. Add a step to ping the WooCommerce REST API.

By clicking the plus sign (+) just below the new email event, let’s add our call back to the WooCommere API to get a full JSON download about the Order.

Search for a service in the left pane then click Woo. Once clicked, you’ll see the dialogue below which leaves you with one stellar option: let’s write some quick Node.js code to retrieve my Order.

In the code window, you’ll notice that Pipedream took the liberty of giving you a basic call, which is much appreciated.

I’ll go ahead and add just a few lines, so my final call to Woo looks like below. Go ahead and paste it into the code window. All I’m doing is pulling the subject line, extracting the Order Number, and adding the Order Number to the template request that was already in Pipedream.

let rawtitle = event.headers["subject"];
let title = encodeURI(rawtitle);

let order = title.replace("New%20Order%20", "");

return await require("@pipedreamhq/platform").axios(this, {
  url: `https://${auths.woocommerce.url}/wp-json/wc/v2/orders`,
  auth: {
    username: `${auths.woocommerce.key}`,
    password: `${auths.woocommerce.secret}`,
  },
  order: order,
})

Importantly, remember to use the dropdown to select the WooCommerce API key you entered earlier.

6. Add a step to add js code for parsing our previous steps.

Next, you can easily replay the email trigger and see if it calls WooCommerce as expected. Click the blue replay button, and note the return data from the WooCommerce API. It’s the nested JSON under > 0 and > 1 >2…

Next, we expand the 0 payload and note that everything about the Order is inside. It’s got the billing information, all the items… everything there is to know about the transaction.

By simply hovering over a value, Pipedream allows you to copy the JSON path of that piece of information. In my case, the key pieces of info that I found in the 0 payload are:

// Phone number 
rawphone = steps.woocommerce.$return_value[0].billing.phone;

// Ready Day (special Woo field in "Custom checkout fields plugin")
day = steps.woocommerce.$return_value[0].meta_data[2].value;


// Ready Time (also from special checkout field plugin)
time = steps.woocommerce.$return_value[0].meta_data[3].value;

Now I’m feeling pretty good about things, because I’ve got the key info I need for the dashboard homepage. And I’ve got the nice email HTML to display when the clerk clicks an Order.

By adding another step in node js, I can neatly lay out all the data I want to display in JSON, just like I would in a normal table.

| Order | 1234 |

| Day | August 8 |

| Time | 1:00pm |

| Phone | 8675309 |

| Item 1 | Sloppy Joe | etc…

To prepare the data for display in the dashboard, I added a node js step and built a table just like the one above.

Here’s the code below, which you can paste-in directly to your Pipedream step. Note that I use let even when const could/should be used. I find this helpful in tools like Pipedream because I often find myself re-introducing variables without realizing I had assigned them previously. I prefer for the code to truly run in straight line.

// pull the key data points from the WooCommerce API response
let rawphone = steps.woocommerce.$return_value[0].billing.phone;
let day = steps.woocommerce.$return_value[0].meta_data[2].value;
let time = steps.woocommerce.$return_value[0].meta_data[3].value;

// quick and dirty fix of various phone # formats, bc frontend validation was a pain
// note that Twilio requires strict phone # formatting. Valid Ex: +12018675309
let fixphone0 = rawphone.replace("+1", "");
let fixphone1 = fixphone0.replace("(", "");
let fixphone2 = fixphone1.replace(")", "");
let fixphone3 = fixphone2.replace(".", "");
let fixphone4 = fixphone3.replace(".", "");
let fixphone5 = fixphone4.replace(" ", "");
let fixphone6 = fixphone5.replace(" ", "");
let fixphone7 = fixphone6.replace("-", "");
let fixphone8 = fixphone7.replace("-", "");
let phone = '+1' + fixphone8;

// to help clerk identify order, pick off first item and tell us if there's other stuff
let firstitem = steps.woocommerce.$return_value[0].line_items[0].name;
let firstitemq = steps.woocommerce.$return_value[0].line_items[0].quantity;
let first = firstitem + ' (x' + firstitemq + ')';
let countsecond = steps.woocommerce.$return_value[0].line_items;
let second = countsecond.length;

// make some changes to the email html for display tweaks
let html = event.html; /*a string*/
let fulltext = encodeURI(html); /*also a string*/
let text1 = fulltext.replace("%3C!DOCTYPE%20html%3E", "");
let text2 = text1.replace("id=%22header_wrapper%22%20style=%22padding:%2036px%2048px;%20display:%20block","id=%22header_wrapper%22%20style=%22padding:%2036px%2048px;%20display:%20none")
let text3 = text2.replace("template_header%22%20style='", "template_header%22%20style='display:none;");
let text4 = text3.replace("style=%22padding:%2048px%2048px%2032px", "style=%22padding:%200px%200px%200px");
let text5 = text4.replace("p%20style=%22margin:%200%200%2016px;%22%3EYou", "p%20style=%22display:none;margin:%200%200%2016px;%22%3EYou");
let text = text5.replace(/1px%20solid/g,"0px%20solid");

//pull out the address in case I need it later
let address = text.substring(
    text.lastIndexOf("A%3Ctable%20id=%22addresses") + 1, 
    text.lastIndexOf("%3Cp%20style=%22margin:%200%200%2016px;%22%3ECongratulations%20on%20the%20sale.")
);

//parse the email to pull out pickup or delivery deets
let pickship = text.substring(
    text.lastIndexOf("%3EShipping:%3C/th%3") + 1, 
    text.lastIndexOf("%3C/td%3E%0A%09%09%09%09%09%3C/tr%3E%0A%09%09%09%09%09%09%09%09%09%09%3Ctr%3E%0A%09%09%09%09%09%09%3Cth%20class=%22td%22%20scope=%22row%22%20colspan=%222%22%20style=%22color:%20#636363;%20border:%200px%20solid%20#e5e5e5;%20vertical-align:%20middle;%20padding:%2012px;%20text-align:%20left;%22%3ETax:")
);

//get ordernum (I could have also carried this var from earlier pipedream step)
let rawtitle = event.headers["subject"];
let title = encodeURI(rawtitle);
let ordernum = title.replace("New%20Order%20", "");

//times of day are harder to sort in dashboard frontend, so assigning weights here 
if (time == '8:30am') {
  weight = "10";
} else if (time == '9:00am') {
  weight = "11";
} else if (time == '9:30am') {
  weight = "12";
} else if (time == '10:00am') {
  weight = "13";
} else if (time == '10:30am') {
  weight = "14";
} else if (time == '11:00am') {
  weight = "15";
} else if (time == '11:30am') {
  weight = "16";
} else if (time == '12:00pm') {
  weight = "17";
} else if (time == '12:30pm') {
  weight = "18";
} else if (time == '1:00pm') {
  weight = "19";
} else if (time == '1:30pm') {
  weight = "20";
} else if (time == '2:00pm') {
  weight = "21";
} else if (time == '2:30pm') {
  weight = "22";
} else if (time == '3:00pm') {
  weight = "23";
} else if (time == '3:30pm') {
  weight = "24";
} else if (time == '4:00pm') {
  weight = "25";
} else if (time == '4:30pm') {
  weight = "26";
} else if (time == '5:00pm') {
  weight = "27";
} else if (time == '5:30pm') {
  weight = "28";
} else if (time == '6:00pm') {
  weight = "29";
} else if (time == '6:30pm') {
  weight = "30";
} else if (time == '7:00pm') {
  weight = "31";
} else if (time == '7:30pm') {
  weight = "32";
} else {
  weight = "0";
};

// take all the fun vars I assigned and make a JSON "table" out of them
let plainjson = { "title" : title, "text" : text, "address" : address, "phone" : phone, "day" : day, "time" : time, "first" : first, "second" : second, "weight" : weight, "caption" : ordernum, "pickship" : pickship, "tags" : "Order", "curstate" : "orderbutton" };


// make sure my json is typed 
let myjson = JSON.stringify(plainjson);

// if you see uri-encoded address in console everything else prob worked 
console.log(address);

//return that json object we made, bc we need it for our tiddlywiki dashboard
return myjson

7. Send the parsed data out.

Now that I’ve organized all of the pertinent Order info from WooCommerce into a neat object (with return myjson), I’ll need to shoot that data over to my TiddlyWiki dashboard.

The plan was to go ahead and just use the Express server I had already created.

There’s one small problem with this approach…

Challenege 3: Auth and protecting my server/endpoint

I like to use Cloudflare to secure most of my node apps, and I want to use it here. It’s important for me to standardize around certain tools so support and management can be streamlined.

In the case of provisioning user access to the dashboard server, authorized folks get a nice magic login code from Cloudflare when they enter their email. This is free up to a team of 50 via the Cloudflare Teams product.

However, if I want to use Teams to authorize an API call, like for my Express server, I have to go up to the $3/user plan.

So, rather than add monthly cost, we can just POST the JSON output from Pipedream to an S3 bucket. Then, with the help of cron, our dashboard server can check the S3 bucket every couple of minutes and download the latest Orders.

Luckily, I really dig microservices, so I have a couple ready-to-go templates in Git to set up the steps in the lower portion of this diagram:

The graphic above represents the final state of my workflow between Woo and my dashboard.

Importantly, note that my Lambda function (and the API Gateway endpoint protecting S3, which is not pictured) were chosen because I had them handy from other projects. If you don’t have these or don’t want to use mine, the easiest thing for you to do at this point is to use Pipedream to write your JSON to S3 directly:

• Setup an IAM user in your AWS account for Pipedream
• Go back to Pipedream “Accounts” section and add your IAM user
• Use built-in Pipedream AWS templates to drop the JSON into a bucket

But, if you want to follow along with my method and use my templates instead, refer to the the aws/ folder in the Git repository to find the Lambda and the Body Mapping Template.

For the Lambda:

All you have to do is create a new Node JS Lambda function in the AWS console and copy-paste the contents of the index.js file into the code window.

For the API Gateway Endpoint

I simply clone existing APIs I have and modify the Body Mapping Template. Using a Mapping Template means that I use API Gateway as another place where I can manipulate the JSON that ultimately gets to my TiddlyWiki dashboard. Refer to aws/body_mapping_template.json for my template. If you have never set up an API in API Gateway before, you’ll want to follow my guide here.

7. Set up pm2 and cron for restarting TiddlyWiki.

So, we’ve played a lot of ping pong with our data, but our dashboard is still without its Orders. The Orders are sitting right there in S3 ripe for the picking, about 10 seconds after they’re placed. Now what?

If you’ve ever wondered about clever ways to “mount” an S3 folder and utilize it on your filesytem, you should consider basic storage replication with rclone (which isn’t an actual “mount,” FYI).

Rclone is a small utility almost exactly like the standard bearer for local replication, rsync. The only difference is that rclone is used to sync a local folder with a “remote” folder, like an S3 bucket.

In the git repository, refer to the server/ folder for ready-to-go shell scripts and quick setup instructions in the server/readme.md. The scripts use pm2 and cron to restart themselves and pull in data every couple of minutes. The dashboard then updates without having to refresh the screen.

• thdash.sh – this is the script to restart TiddlyWiki, which is required to load new Orders.
• rclonenew.sh – this is the rclone script to poll S3 for new Orders every few minutes.
• dater.sh – this script overwrites the Wiki’s tiddlers/DayCheck.tid Tiddler, which tells the dash what day it is.

The Bliss – Integrating Twilio SMS to contact customers

Alright!

Now that we’ve followed the Readmes in git, we’ve got a WooCommerce dashboard that neatly drops a beautiful nugget of JSON onto our dashboard whenever a new Order comes in.

But now that we have the Order, we want to be able to communicate with the customer and update them on the status. Maybe “Give us an extra 10 minutes,” or “No soup for you!”

To set up your WooCommerce dashboard with Twilio, all you have to do is enter a few pieces of information from your Twilio account into your Wiki dashboard. Then you’ll be ready to send texts.

1. In a text editor before launching your Wiki, swap out the “Basic” Auth Code and the “From” number.

In the dashboard’s files from git, which you installed on your server, use a text editor to navigage to server/tiddlers/$__plugins_OokTech_SubmitForm_action-submitform.js.

The ‘From’ Number

To get your “From” number, first sign up for a Twilio account. When you set up a demo account for SMS, one of the first things you do is select a From number. Then navigate to ‘Manage Numbers’ in the left pane to view your number.

When you enter your number in tiddlers/$__plugins_OokTech_SubmitForm_action-submitform.js, make sure your number includes your country code (1 for USA), and excludes the plus sign (+) prefix, since I add it here for you with %2B.

Note that, if your Twilio account is brand new, you will start with a demo account. The demo account can only send a text to an authorized test phone (ie, your cell phone). Once testing is complete, you can switch from a demo account to a paid account. Twilio will then allow a limited number of texts to whomever, until your account is manually verified by Twitter staff.

The Basic Auth Code

To get the other code that gets loaded into tiddlers/$__plugins_OokTech_SubmitForm_action-submitform.js, navigate to your Twilio dashboard and, in the left pane, click the circle with the 3 dots. Then scroll down and select “API Explorer.”

Once you reach the API explorer, you will see that you have the ability to grab pre-built commands for various Twilio API methods. Be sure to select “Programmable Messaging” from the dropbox on top.

Since we want the ability to send POST requests from our HTML dashboard, we’ll select the POST request and examine the samples inside. We can even send ourselves a test right from this tool.

Go ahead and select the POST request, and note the URL endpoint that is provided in the CURL request. Copy this entire URL.

You’ll note that the URL looks something like this:

https://api.twilio.com/2010-04-01/Accounts/xxx_YOUR_SECRET_CODE_xxx/Messages.json

To get the Basic Auth Code for the Wiki file, you just a need a Base64 encoded version of YOUR_SECRET_CODE from the URL. To get this, all you need to do is copy the code and paste it into a tool like base64encode.org. When you click “Encode,” you’ll get a longer encoded output, which is a new representation of the same secret. It will look something like this:

QFALJFHVDSIHFKJSFHKJSRFGHKJSRGNVNLSASPAPOERKJFSLKNFw==

That output is the AUTH_CODE you need to enter into the tiddlers/$__plugins_OokTech_SubmitForm_action-submitform.js after the word “Basic.”

2. In a text editor, update the Wiki with your own Twilio endpoint URL.

Finally, taking our Twilio endpoint URL from the previous step.

https://api.twilio.com/2010-04-01/Accounts/xxx_YOUR_SECRET_CODE_xxx/Messages.json

We can update our Wiki to use our special URL. To do so, use a text editor to navigate to tiddlers/mymacros and edit the TWILIO_URL definition.

You’re all set!

Now, to wrap it all up, you can restart your TiddlyWiki server and send an Order on through!

pm2 list

//if the thdash wiki is process 0 in the pm2 list command
pm2 restart 0 

Now go ahead and place an order, and voila! You’ve got a WooCommerce dashboard with SMS capability.

You can edit the content of the text messages by searching for them right in the dashboard itself. First open the TiddlyWiki sidebar by clicking the chevron button in the top-right. Then search for “Msg-” in the search bar. Click any Tiddler to view it and edit its content. I’ll try to build some clean forms for these in the future.

Lunch is served!

---

If you liked this post, follow me on Twitter @philwonski so you know when I post similar content. You can see past posts by me here.

There’s also lots of great (and less esoteric) content being produced by our team on a regular basis — be sure to find us on LinkedIn, and sign up to our newsletter in the footer of this page.