Daurnimator's Hovel

Instant postgres

2016-05-07T11:10:27+00:00

Sometimes I need a temporary postgres to try something out. I keep this script around as instantpg.sh:

#!/bin/bash
trap "rm -rf -- $PGDATA" EXIT
export PGDATA="$(mktemp -d)" PGPORT=1234
initdb
echo "Connect with: psql -p \"$PGPORT\" -h localhost -d postgres"
postgres -F -k ""

Migrating from luasocket to lua-http

2016-04-06T01:17:18+00:00

I saw https://github.com/brunoos/luasec/issues/72#issuecomment-205328635 and couldn’t resist writing the linked code to use lua-http instead.

As the code was originally using luasocket’s http interface, it was straightforward to convert it to lua-http’s http.compat.socket module. This compatability interface provides the same API as luasocket’s socket.http and luasec’s ssl.https modules.


local http = require "http.compat.socket" -- require "socket.http"
local https = http -- require "ssl.https"
local ltn12 = require "ltn12"
local string_sub = string.sub
local table_concat = table.concat
local function make_request(url, timeout)
    http.TIMEOUT = timeout
    http.USERAGENT = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36"
    local b, c, h, s
    local tbody = {}
    local https_opt = {
        url = url,
        protocol = "tlsv1",
        verify = "none",
        headers = {
            ["Accept"] = "*/*",
            ["Accept-Language"] = "sk;q=0.8,en-US,en;q=0.6,cs;q=0.4",
            ["Accept-Charset"] = "UTF-8;q=0.8,*;q=0.7",
        },
        sink = ltn12.sink.table(tbody),
        redirect = false,
    }
    local http_opt = {
        url = url,
        headers = {
            ["Accept"] = "*/*",
            ["Accept-Language"] = "sk;q=0.8,en-US,en;q=0.6,cs;q=0.4",
            ["Accept-Charset"] = "UTF-8;q=0.8,*;q=0.7",
        },
        sink = ltn12.sink.table(tbody),
        redirect = false,
    }
    if string_sub(url, 1, 5) ~= "https" then
        _, c, h, s = http.request(http_opt)
    else
        _, c, h, s = https.request(https_opt)
    end
    -- make headers keys lowercase
    if h ~= nil then
        local h_tmp = {}
        for k, v in pairs(h) do h_tmp[k:lower()] = v end
        h = h_tmp
    end
    -- concat body parts
    b = table_concat(tbody)
    return {body = b, code = c, headers = h, status = s}
end

However, this function can be rewritten much more nicely using the http.request module.


local http_req = require "http.request"
local h1_reason_phrases = require "http.h1_reason_phrases"
local function make_request(url, timeout)
    local r = http_req.new_from_uri(url)
    r.headers:upsert("useragent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36")
    r.headers:upsert("accept", "*/*")
    r.headers:upsert("accept-language", "sk;q=0.8,en-US,en;q=0.6,cs;q=0.4")
    r.headers:upsert("accept-charset", "UTF-8;q=0.8,*;q=0.7")
    r.follow_redirects = false
    local headers, stream = r:go(timeout)
    if headers == nil then
        return nil, stream
    end
    local b, err = stream:get_body_as_string(timeout) -- XXX: use a deadline instead of a timeout?
    stream:shutdown() -- shutdown ASAP to free resources
    if b == nil then
        return nil, err
    end
    local c = headers:get(":status")
    local s = h1_reason_phrases[c] -- look up reason phrase for code
    c = tonumber(c, 10) or c -- the code might not be numeric
    -- convert from headers object to unordered table of key/value pairs
    local h = {}
    for name in headers:each() do
        if name ~= ":status" and h[name] == nil then
            h[name] = headers:get_comma_separated(name)
        end
    end
    return {body = b, code = c, headers = h, status = s}
end

If the user has the flexibility to change the API of their make_request function, they may wish to use the lua-http http.headers object directly instead of transforming it to code + table of header name => value pairs.

Lua lib to send SMS with Telstra’s SMS API

2016-02-28T11:14:52+00:00

Lua lib to send SMS with Telstra's SMS API

Wrote a lua library wrapper around the Telstra SMS API.

lpeg_patterns v0.2

2015-12-13T14:36:46+00:00

I’m happy to announce the 0.2 release of lpeg_patterns.

“lpeg_patterns” is a collection of patterns I’ve written for various widely used formats. Current sub-modules are: IPv4, IPv6, email addresses, phone numbers, uri.

This release includes:

Fixed parsing of IPv6 addresses (thanks spc)
IPv6 zone support
Stricter uri matching (scheme is now compulsory)
“reference” (i.e. relative) URI matching

Homepage: https://github.com/daurnimator/lpeg_patterns

It’s available via luarocks: https://luarocks.org/modules/daurnimator/lpeg_patterns

Testing pre-commit with git

2015-12-04T10:24:27+00:00

It’s great to run tests on your code before you commit it. It’s even better to make that happen automatically!

git lets you run a script before a commit succeeds by creating what is known as a “pre-commit” hook; it’s simply an executable located at .git/hooks/pre-commit.

However, there are a few gotchas: by default, git will just run it against your current (possibly dirty) checkout. To make sure you’re actually testing the code you’re about to commit, we can stash your other changes while the tests run.

This brings up another issue: that popping a git stash that includes an index will often result in conflicts; we can solve this by using git reset --hard before we pop from the stash.

One last thing: we want to recover to the original directory state no matter what happens (e.g. maybe our test suite itself fails); so we use a bash trap to ensure that our stash popping happens no-matter the exit path.

With that all said, here is what I use as a pre-commit hook for my lua projects:

#!/bin/bash
set -eufo pipefail

if [ -n "$(git status -z)" ]; then
    git stash -q --keep-index --include-untracked
    trap "git reset -q --hard && git stash pop -q --index" EXIT
fi

echo "## Running luacheck"
luacheck .
echo

echo "## Running tests with busted"
busted
echo

The Codist: Software Estimation Is A Crock

2015-08-20T03:13:10+00:00

The Codist: Software Estimation Is A Crock

Fifo.lua 0.2 Released

2015-08-04T13:09:41+00:00

Fifo.lua 0.2 Released

Today I’m releasing verion 0.2 of “fifo.lua”. A lua library that provides a fifo of lua objects.

Changes since 0.1

:setempty now returns the fifo so that initialisation is simplified
:peek returns 2nd value indicating if there was a value at the given index
Remove undocumented :iter and :foreach methods
Added __len metamethod as alias for :length method (only available in 5.2+)
Fixed bug in :remove method where the last element could not be removed
Stricter argument checking
Move documentation out of README into a pandoc compatible markdown file
Dropped Lua 5.0 support
Improved performance (especially in LuaJIT)

daurnimator/lua-http

2015-08-04T12:39:50+00:00

daurnimator/lua-http

Over the last few weeks, I’ve been working on new lua library in my free time.

Announcing: “lua-http”.

It’s a fresh take on a http library for lua. Designed it with HTTP2 in mind from the start, it caters for both clients and servers.

All network operations are non-blocking, yet also work outside of a coroutine. This allows for lua-http to be used in any lua project, application or script.

daurnimator/multi-protocol.lua

2015-07-17T14:46:37+00:00

daurnimator/multi-protocol.lua

This snippet demonstrates listening for multiple protocol signatures on the one TCP port. I left off concurrency to make it easier to understand.

The TLS path will only work with the newly commited cqueues :starttls pushback, which runs userspace buffered data through the incoming BIO before reading off the socket.

A couple of the protocol patterns I stole from prosody (where a similar idea is done via ‘mod_net_multiplex’).

C bug of the day

2015-07-10T06:35:11+00:00

C bug of the day