Some Go Web Development Notes

In the past few weeks, I have spent a lot of time developing a website in Go that may never come out, but I have learned something in the process and want to document it. Here are some of my notes:

Go 1.22 now has better routing

I have never had the motivation to learn any Go routing libraries (such as gorilla/mux, chi, etc.), so I have been manually handling all the routing, like this:

http.HandleFunc("/", func(w http.ResponseWriter, r \*http.Request) {
    if r.URL.Path != "/" {
        http.NotFound(w, r)
        return
    }
    fmt.Fprintf(w, "Welcome to the home page!")
})

http.HandleFunc("/hello", func(w http.ResponseWriter, r \*http.Request) {
    fmt.Fprintf(w, "Hello!")
})

But obviously starting from Go 1.22[1]Go now has better support for routing in the standard library, so the above code can be rewritten as follows:

http.HandleFunc("GET /", func(w http.ResponseWriter, r \*http.Request) {
    fmt.Fprintf(w, "Welcome to the home page!")
})

http.HandleFunc("GET /hello", func(w http.ResponseWriter, r \*http.Request) {
    fmt.Fprintf(w, "Hello!")
})

However, it also requires a login middleware, so it may be more like this, using arequireLoginMiddleware:

mux := http.NewServeMux()
mux.HandleFunc("GET /", requireLogin(homePage))
mux.HandleFunc("GET /hello", requireLogin(helloPage))

A trap of built-in routers: redirects with trailing slashes

The annoying trap I encountered is: if I/records/Create a route, then/recordsThe request will be redirected[2]reach/records/。

I have encountered a problem/recordsSending POST requests will redirect to/records/The GET request broke the POST request because it removed the request body. Fortunately, Xe Iaso wrote a blog post about the exact same issue[3]This makes debugging easier.

I think the solution to this problem is to use something likePOST /recordsThis API endpoint, rather thanPOST /records/This seems to be a more normal design as well.

Sqlc automatically generates code for my database query

I’m a little tired of writing so much template code for SQL queries, but I don’t want to learn ORM because I know what SQL queries I want to write, and I don’t want to learn the conventions of ORM that convert things into SQL queries.

But later on, I discovered SQLC[4]It can perform such queries:

\-- name: GetAuthor :one
SELECT \* FROM authors
WHERE id \= $1 LIMIT 1;

\-- name: ListAuthors :many
SELECT \* FROM authors
ORDER BY name;

\-- name: CreateAuthor :one
INSERT INTO authors (
  name, bio
) VALUES (
  $1, $2
)
RETURNING \*;

\-- name: DeleteAuthor :exec
DELETE FROM authors
WHERE id \= $1;

Compile into Go code like this:

type Author struct {
	ID   int64
	Name string
	Bio  string
}

func (q \*Queries) GetAuthor(ctx context.Context, id int64) (Author, error) {
	row := q.db.QueryRowContext(ctx, "SELECT id, name, bio FROM authors WHERE id = $1 LIMIT 1", id)
	var i Author
	err := row.Scan(&i.ID, &i.Name, &i.Bio)
	return i, err
}

func (q \*Queries) ListAuthors(ctx context.Context) (\[\]Author, error) {
	rows, err := q.db.QueryContext(ctx, "SELECT id, name, bio FROM authors ORDER BY name")
	if err != nil {
		return nil, err
	}
	defer rows.Close()
	var items \[\]Author
	for rows.Next() {
		var i Author
		if err := rows.Scan(&i.ID, &i.Name, &i.Bio); err != nil {
			return nil, err
		}
		items = append(items, i)
	}
	if err := rows.Close(); err != nil {
		return nil, err
	}
	if err := rows.Err(); err != nil {
		return nil, err
	}
	return items, nil
}

func (q \*Queries) CreateAuthor(ctx context.Context, arg CreateAuthorParams) (Author, error) {
	row := q.db.QueryRowContext(ctx, "IN SERT INTO authors (name, bio) VALUES ($1, $2) RETURNING id, name, bio", arg.Name, arg.Bio)
	var i Author
	err := row.Scan(&i.ID, &i.Name, &i.Bio)
	return i, err
}

func (q \*Queries) DeleteAuthor(ctx context.Context, id int64) error {
	\_, err := q.db.ExecContext(ctx, "DELETE FROM authors WHERE id = $1", id)
	return err
}

The reason why I like this approach is that if I’m not sure what Go code to write for a given SQL query, I can simply write the query I want, read the generated function, and it will tell me exactly how to call it. For me, this is much easier than trying to flip through the documentation of ORM to figure out how to construct the SQL query I want.

Read Brandur’s SQLC notes for 2024[5]It also gave me some confidence that this method is feasible in my mini program. This article provides a very useful example of how to use CASE statements to conditionally update fields in a table (for example, if you have a table with 20 columns and only want to update 3 of them).

SQLite Tips

Someone on Mastodon sent me an article titled ‘Optimizing SQLite on a Server’[6]The article link. My project is small and I don’t really care about performance, but my main takeaway is:

• There is one specifically designed for write in Objects in the database and running on themdb.SetMaxOpenConns(1). I learned through difficult means that if I don’t do this, when two threads try to write to the database at the same time, I will getSQLITE_BUSYError.
• If I want to make reading faster, I can have two independent database objects, one for writing and one for reading

There are more seemingly useful techniques in that article (such as “COUNT queries are slow” and “using STRICT tables”), but I haven’t tried them yet.

Also, sometimes if I have two tables, I know I never need to do anything between themJOINI will put them in separate databases so that I can connect them independently.

Go 1.19 introduces a method for setting GC memory limits

I run all my Go projects in a virtual machine with relatively less memory, such as 256MB or 512MB. I have encountered a problem where my application is constantly being killed by OOM, which is very confusing - do I have a memory leak? What’s going on?

After some Google searches, I realized that maybe I don’t have a memory leak, or maybe I just need to reconfigure the garbage collector! By default (according to the Go garbage collector guide)[7]）Go’s garbage collector will allocate up to the current heap size of the memory allocated by the application 2 times 。

Mess With DNS[8]The basic heap size of is about 170MB, and the currently available memory on the virtual machine is about 160MB, so if its memory doubles, it will be killed by OOM.

In Go 1.19, they added a method to tell Go ‘Hey, if the application starts using so much memory, run GC once’. So I set the GC memory limit to 250MB, which seems to have reduced the frequency of OOM killing applications:

import "runtime/debug"

func main() {
    debug.SetMemoryLimit(250 \* 1024 \* 1024) // 250MB
    // ...
}

Some reasons why I like to use Go to create websites

In the past 4 years or so, I have intermittently used Go to create small websites (such as nginx playground)[9]）This really suits me. I think I like it because:

• There is only one static binary file, all I need to do is copy this binary file. If there are static files, I can use embed[10]Embed them into binary files.
• There is a built-in web server that can be used in production environments, so I don’t need to configure WSGI or anything else to make it work. I can put it directly in Caddy[11]Run it later or on fly.io.
• The toolchain of Go is very easy to install, I can directly install itapt-get install golang-goOr in other ways, and thengo buildI can build my project
• I feel like there are very few things to remember when starting to send HTTP responses - basically only things likeServe(w http.ResponseWriter, r *http.Request)These functions read requests and send responses. If I need to remember certain details of how it was completed, I just need to read the function!
• andnet/httpIn the standard library, so you can start making websites without installing any libraries. I am truly grateful for this.
• Go is a fairly systematic language, so if I need to run itioctlOr something similar, that’s easy to achieve

Overall, everything about it feels like it makes the project easy to get started in 5 days, give up for 2 years, and then come back to continue coding without encountering too many problems.

In contrast, I have tried learning Rails a few times and I really want to fall in love with it - I have created several toy websites using Rails and it always feels like a magical experience. But in the end, when I returned to these projects, I couldn’t remember how anything worked, and I always gave up in the end. For me, returning to my Go projects filled with repetitive template code feels easier because at least I can read the code and figure out how it works.

Something I haven’t figured out yet

There are some things I haven’t done yet in Go:

• Rendering HTML templates: Usually my Go server is just an API, and I use Vue to create single page applications as the frontend. I have extensively used it in Hugohtml/templateI have been using it to do this blog for the past 8 years, but I am still unsure how I feel about it.
• I have never created a real login system, usually my server has no users at all.
• I have never attempted to implement CSRF before

Overall, I am not sure how to implement security sensitive features, so I will not start projects that require login/CSRF and other functionalities. I think this is where the framework would be helpful.

It’s cool to see that Go keeps adding new features

The two Go features I mentioned in this article（GOMEMLIMITBoth routing and routing were added in the past few years, and I didn’t notice them when they were released. This makes me feel that I should pay closer attention to the release notes of the new Go version.

Reference link

1. Starting from Go 1.22: https://go.dev/blog/routing-enhancements
2. Will be redirected: https://pkg.go.net/net/http#hdr -Trailing_slash_redirection-ServeMux
3. Xe Iaso wrote a blog post about the exact same issue: https://xeiaso.net/blog/go-servemux-slash-2021-11-04/
4. sqlc: https://sqlc.dev/
5. Brandur’s SQLC notes for 2024: https://brandur.org/fragments/sqlc-2024
6. Optimize SQLite on the server: https://kerkour.com/sqlite-for-servers
7. Go Garbage Collector Guide: https://tip.golang.org/doc/gc-guide
8. Mess With DNS: https://messwithdns.net/
9. nginx playground: https://nginx-playground.wizardzines.com/
10. embed: https://pkg.go.dev/embed
11. Caddy: https://caddyserver.com/