Migrating from luasocket to lua-http

I saw https://github.com/brunoos/luasec/issues/72#issuecomment-205328635 and couldn’t resist writing the linked code to use lua-http instead.

As the code was originally using luasocket’s http interface, it was straightforward to convert it to lua-http’s http.compat.socket module. This compatability interface provides the same API as luasocket’s socket.http and luasec’s ssl.https modules.


local http = require "http.compat.socket" -- require "socket.http"
local https = http -- require "ssl.https"
local ltn12 = require "ltn12"
local string_sub = string.sub
local table_concat = table.concat
local function make_request(url, timeout)
    http.TIMEOUT = timeout
    http.USERAGENT = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36"
    local b, c, h, s
    local tbody = {}
    local https_opt = {
        url = url,
        protocol = "tlsv1",
        verify = "none",
        headers = {
            ["Accept"] = "*/*",
            ["Accept-Language"] = "sk;q=0.8,en-US,en;q=0.6,cs;q=0.4",
            ["Accept-Charset"] = "UTF-8;q=0.8,*;q=0.7",
        },
        sink = ltn12.sink.table(tbody),
        redirect = false,
    }
    local http_opt = {
        url = url,
        headers = {
            ["Accept"] = "*/*",
            ["Accept-Language"] = "sk;q=0.8,en-US,en;q=0.6,cs;q=0.4",
            ["Accept-Charset"] = "UTF-8;q=0.8,*;q=0.7",
        },
        sink = ltn12.sink.table(tbody),
        redirect = false,
    }
    if string_sub(url, 1, 5) ~= "https" then
        _, c, h, s = http.request(http_opt)
    else
        _, c, h, s = https.request(https_opt)
    end
    -- make headers keys lowercase
    if h ~= nil then
        local h_tmp = {}
        for k, v in pairs(h) do h_tmp[k:lower()] = v end
        h = h_tmp
    end
    -- concat body parts
    b = table_concat(tbody)
    return {body = b, code = c, headers = h, status = s}
end

However, this function can be rewritten much more nicely using the http.request module.


local http_req = require "http.request"
local h1_reason_phrases = require "http.h1_reason_phrases"
local function make_request(url, timeout)
    local r = http_req.new_from_uri(url)
    r.headers:upsert("useragent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36")
    r.headers:upsert("accept", "*/*")
    r.headers:upsert("accept-language", "sk;q=0.8,en-US,en;q=0.6,cs;q=0.4")
    r.headers:upsert("accept-charset", "UTF-8;q=0.8,*;q=0.7")
    r.follow_redirects = false
    local headers, stream = r:go(timeout)
    if headers == nil then
        return nil, stream
    end
    local b, err = stream:get_body_as_string(timeout) -- XXX: use a deadline instead of a timeout?
    stream:shutdown() -- shutdown ASAP to free resources
    if b == nil then
        return nil, err
    end
    local c = headers:get(":status")
    local s = h1_reason_phrases[c] -- look up reason phrase for code
    c = tonumber(c, 10) or c -- the code might not be numeric
    -- convert from headers object to unordered table of key/value pairs
    local h = {}
    for name in headers:each() do
        if name ~= ":status" and h[name] == nil then
            h[name] = headers:get_comma_separated(name)
        end
    end
    return {body = b, code = c, headers = h, status = s}
end

If the user has the flexibility to change the API of their make_request function, they may wish to use the lua-http http.headers object directly instead of transforming it to code + table of header name => value pairs.

lpeg_patterns v0.2

I’m happy to announce the 0.2 release of lpeg_patterns.

“lpeg_patterns” is a collection of patterns I’ve written for various widely used formats. Current sub-modules are: IPv4, IPv6, email addresses, phone numbers, uri.

This release includes:

  • Fixed parsing of IPv6 addresses (thanks spc)
  • IPv6 zone support
  • Stricter uri matching (scheme is now compulsory)
  • “reference” (i.e. relative) URI matching

Homepage: https://github.com/daurnimator/lpeg_patterns

It’s available via luarocks: https://luarocks.org/modules/daurnimator/lpeg_patterns

Testing pre-commit with git

It’s great to run tests on your code before you commit it. It’s even better to make that happen automatically!

git lets you run a script before a commit succeeds by creating what is known as a “pre-commit” hook; it’s simply an executable located at .git/hooks/pre-commit.

However, there are a few gotchas: by default, git will just run it against your current (possibly dirty) checkout. To make sure you’re actually testing the code you’re about to commit, we can stash your other changes while the tests run.

This brings up another issue: that popping a git stash that includes an index will often result in conflicts; we can solve this by using git reset --hard before we pop from the stash.

One last thing: we want to recover to the original directory state no matter what happens (e.g. maybe our test suite itself fails); so we use a bash trap to ensure that our stash popping happens no-matter the exit path.

With that all said, here is what I use as a pre-commit hook for my lua projects:

#!/bin/bash
set -eufo pipefail

if [ -n "$(git status -z)" ]; then
    git stash -q --keep-index --include-untracked
    trap "git reset -q --hard && git stash pop -q --index" EXIT
fi

echo "## Running luacheck"
luacheck .
echo

echo "## Running tests with busted"
busted
echo