Go Web App Example - Entry Point, File Structure, Models, and Routes

04 Jan, 2016

There are a few frameworks for Go that make it easier to build a web Application. Instead of using one of the those frameworks, I wanted to stick with the Go mentality of only using individual packages and build an example model-view-controller (MVC) web application with authentication. Keep in mind, this is just one of the many ways to structure your web application.

All the code and screenshots are available on GitHub: https://github.com/josephspurrier/gowebapp

File Structure

These are the folders and files at the root:

config/       - application settings and database schema
static/       - location of statically served files like CSS and JS
template/     - HTML templates

vendor/app/controller/   - page logic organized by HTTP methods (GET, POST)
vendor/app/model/        - database queries
vendor/app/route/        - route information and middleware
vendor/app/shared/       - packages for templates, MySQL, cryptography, sessions, and json

gowebapp.db   - SQLite database
gowebapp.go   - application entry point

External Packages

The external packages I used:

github.com/gorilla/context              - registry for global request variables
github.com/gorilla/sessions             - cookie and filesystem sessions
github.com/go-sql-driver/mysql          - MySQL driver
github.com/haisum/recaptcha             - Google reCAPTCHA support
github.com/jmoiron/sqlx                 - MySQL general purpose extensions
github.com/josephspurrier/csrfbanana    - CSRF protection for gorilla sessions
github.com/julienschmidt/httprouter     - high performance HTTP request router
github.com/justinas/alice               - middleware chaining
github.com/mattn/go-sqlite3             - SQLite driver
golang.org/x/crypto/bcrypt              - password hashing algorithm

Application Entry Point

My goal with the main package, gowebapp.go, was to only do a few things:

Define the application settings structs
Read the JSON configuration file and pass those settings to each package
Start the HTTP listener

By using this strategy, it’s easy to add or remove components as you build out your application because all the components are configured in one place.

All the imported packages are either from the standard library or a wrapper.

package main

import (
	"encoding/json"
	"log"
	"os"
	"runtime"

	"app/route"
	"app/shared/database"
	"app/shared/email"
	"app/shared/jsonconfig"
	"app/shared/recaptcha"
	"app/shared/server"
	"app/shared/session"
	"app/shared/view"
	"app/shared/view/plugin"
)

The application settings are defined in the configuration struct and then stored in the config variable.

// config the settings variable
var config = &configuration{}

// configuration contains the application settings
type configuration struct {
	Database  database.Info  	      `json:"Database"`
	Email     email.SMTPInfo          `json:"Email"`
	Recaptcha recaptcha.Info		  `json:"Recaptcha"`
	Server    server.Server           `json:"Server"`
	Session   session.Session         `json:"Session"`
	Template  view.Template           `json:"Template"`
	View      view.View               `json:"View"`
}

// ParseJSON unmarshals bytes to structs
func (c *configuration) ParseJSON(b []byte) error {
	return json.Unmarshal(b, &c)
}

The runtime settings and flags are defined in the init() func. The components are passed the settings from the config.json file in the main() func. The server is then started so the application is accessible via a web browser.

func init() {
	// Verbose logging with file name and line number
	log.SetFlags(log.Lshortfile)

	// Use all CPU cores
	runtime.GOMAXPROCS(runtime.NumCPU())
}

func main() {
	// Load the configuration file
	jsonconfig.Load("config"+string(os.PathSeparator)+"config.json", config)

	// Configure the session cookie store
	session.Configure(config.Session)

	// Connect to database
	database.Connect(config.Database)

	// Configure the Google reCAPTCHA prior to loading view plugins
	recaptcha.Configure(config.Recaptcha)

	// Setup the views
	view.Configure(config.View)
	view.LoadTemplates(config.Template.Root, config.Template.Children)
	view.LoadPlugins(
		plugin.TagHelper(config.View),
		plugin.NoEscape(),
		plugin.PrettyTime(),
		recaptcha.Plugin())

	// Start the listener
	server.Run(route.LoadHTTP(), route.LoadHTTPS(), config.Server)
}

Shared Packages

I wanted the application components to be as decoupled as possible. Each component would be in its own package with a struct that defined its settings. I didn’t want a global registry or a generic container because that creates too many dependencies. I designed it so when the application starts, a single JSON config file is parsed and then passed to a Configure() or Load() func in each package. Many of the packages in the vendor/app/shared/ folder are just wrappers for an external packages. Recently, I decided to move the majority of the Go code to the app folder inside the vendor folder so my own github path is not littered throughout the many imports.

This architecture provides the following benefits:

Each package in the vendor/app/shared/ folder only imports packages from the standard library or an external package so it’s easy to reuse the package in other applications
When adding configurable settings to each package, they only need to be added in two places: the config.json file and the package itself
If the API of an external package changes, it’s easy to update the wrapper without modifying any code in your core application

Let’s take a look at the session package step by step.

The package only imports from the standard library and an external package.

package session

import (
	"net/http"

	"github.com/gorilla/sessions"
)

The struct called Session defines the settings for the package which are readable from a JSON file. A few of the variables are stored in package level variables and are publicly accessible.

var (
	// Store is the cookie store
	Store *sessions.CookieStore
	// Name is the session name
	Name string
)

// Session stores session level information
type Session struct {
	Options   sessions.Options `json:"Options"`   // Pulled from: http://www.gorillatoolkit.org/pkg/sessions#Options
	Name      string           `json:"Name"`      // Name for: http://www.gorillatoolkit.org/pkg/sessions#CookieStore.Get
	SecretKey string           `json:"SecretKey"` // Key for: http://www.gorillatoolkit.org/pkg/sessions#CookieStore.New
}

The Configure() func is passed the struct instead of individual parameters so no code needs to be changed outside of the package (except in the JSON file) when the Session struct is changed.

// Configure the session cookie store
func Configure(s Session) {
	Store = sessions.NewCookieStore([]byte(s.SecretKey))
	Store.Options = &s.Options
	Name = s.Name
}

The package is used by calling the Instance() func so the core application doesn’t have to reference the gorilla/sessions package directly.

// Session returns a new session, never returns an error
func Instance(r *http.Request) *sessions.Session {
	session, _ := Store.Get(r, Name)
	return session
}

Models

All the models are stored in the vendor/app/model/ folder under one package. The application supports Bolt, MySQL, and MongoDB, but can be easily changed to use another type of database. The example application only has a single model called user.go. The model package is the only place SQL code is written. The file is structured into two parts:

Structs for each table
Funcs for each query

It’s a good idea to make sure the types in the struct match the types in the database.

// User table contains the information for each user
type User struct {
	ObjectID  bson.ObjectId `bson:"_id"`
	ID        uint32        `db:"id" bson:"id,omitempty"` // Don't use Id, use UserID() instead for consistency with MongoDB
	FirstName string        `db:"first_name" bson:"first_name"`
	LastName  string        `db:"last_name" bson:"last_name"`
	Email     string        `db:"email" bson:"email"`
	Password  string        `db:"password" bson:"password"`
	StatusID  uint8         `db:"status_id" bson:"status_id"`
	CreatedAt time.Time     `db:"created_at" bson:"created_at"`
	UpdatedAt time.Time     `db:"updated_at" bson:"updated_at"`
	Deleted   uint8         `db:"deleted" bson:"deleted"`
}

// UserStatus table contains every possible user status (active/inactive)
type UserStatus struct {
	ID        uint8     `db:"id" bson:"id"`
	Status    string    `db:"status" bson:"status"`
	CreatedAt time.Time `db:"created_at" bson:"created_at"`
	UpdatedAt time.Time `db:"updated_at" bson:"updated_at"`
	Deleted   uint8     `db:"deleted" bson:"deleted"`
}

The query is stored in a func and clearly named so there is no confusion as to what the query does. To make the app support multiple database types, the code for each database type is contained in each func.

// UserByEmail gets user information from email
func UserByEmail(email string) (User, error) {
	var err error

	result := User{}

	switch database.ReadConfig().Type {
	case database.TypeMySQL:
		err = database.SQL.Get(&result, "SELECT id, password, status_id, first_name FROM user WHERE email = ? LIMIT 1", email)
	case database.TypeMongoDB:
		if database.CheckConnection() {
			session := database.Mongo.Copy()
			defer session.Close()
			c := session.DB(database.ReadConfig().MongoDB.Database).C("user")
			err = c.Find(bson.M{"email": email}).One(&result)
		} else {
			err = ErrUnavailable
		}
	case database.TypeBolt:
		err = database.View("user", email, &result)
		if err != nil {
			err = ErrNoResult
		}
	default:
		err = ErrCode
	}

	return result, standardizeError(err)
}

// UserCreate creates user
func UserCreate(firstName, lastName, email, password string) error {
	var err error

	now := time.Now()

	switch database.ReadConfig().Type {
	case database.TypeMySQL:
		_, err = database.SQL.Exec("INSERT INTO user (first_name, last_name, email, password) VALUES (?,?,?,?)", firstName,
			lastName, email, password)
	case database.TypeMongoDB:
		if database.CheckConnection() {
			session := database.Mongo.Copy()
			defer session.Close()
			c := session.DB(database.ReadConfig().MongoDB.Database).C("user")

			user := &User{
				ObjectID:  bson.NewObjectId(),
				FirstName: firstName,
				LastName:  lastName,
				Email:     email,
				Password:  password,
				StatusID:  1,
				CreatedAt: now,
				UpdatedAt: now,
				Deleted:   0,
			}
			err = c.Insert(user)
		} else {
			err = ErrUnavailable
		}
	case database.TypeBolt:
		user := &User{
			ObjectID:  bson.NewObjectId(),
			FirstName: firstName,
			LastName:  lastName,
			Email:     email,
			Password:  password,
			StatusID:  1,
			CreatedAt: now,
			UpdatedAt: now,
			Deleted:   0,
		}

		err = database.Update("user", user.Email, &user)
	default:
		err = ErrCode
	}

	return standardizeError(err)
}

Routes

Each of the routes are defined in route.go. I decided to use julienschmidt/httprouter for the speed and then justinas/alice for chaining access control lists (ACLs) to the controller funcs with the main logic for each page. All the middleware is also defined in one place.

I’m going to skip the imports and show how the middleware and routes are combined into HTTP and HTTPS routes. There is also an easy way to redirect HTTP to HTTPS if you would like.

// Load the routes and middleware
func Load() http.Handler {
	return middleware(routes())
}

// Load the HTTP routes and middleware
func LoadHTTPS() http.Handler {
	return middleware(routes())
}

// Load the HTTPS routes and middleware
func LoadHTTP() http.Handler {
	return middleware(routes())

	// Uncomment this and comment out the line above to always redirect to HTTPS
	//return http.HandlerFunc(redirectToHTTPS)
}

// Optional method to make it easy to redirect from HTTP to HTTPS
func redirectToHTTPS(w http.ResponseWriter, req *http.Request) {
	http.Redirect(w, req, "https://"+req.Host, http.StatusMovedPermanently)
}

I’ll show a few of the routes. The 404 handler and static files are set along with the home page. The different types of HTTP requests like GET and POST are defined separately and each one has an ACL that controls who can access the page.

func routes() *httprouter.Router {
	r := httprouter.New()

	// Set 404 handler
	r.NotFound = alice.
		New().
		ThenFunc(controller.Error404)

	// Serve static files, no directory browsing
	r.GET("/static/*filepath", hr.Handler(alice.
		New().
		ThenFunc(controller.Static)))

	// Home page
	r.GET("/", hr.Handler(alice.
		New().
		ThenFunc(controller.Index)))

	// Login
	r.GET("/login", hr.Handler(alice.
		New(acl.DisallowAuth).
		ThenFunc(controller.LoginGET)))
	r.POST("/login", hr.Handler(alice.
		New(acl.DisallowAuth).
		ThenFunc(controller.LoginPOST)))
	r.GET("/logout", hr.Handler(alice.
		New().
		ThenFunc(controller.Logout)))
...
}

The middleware is then added to the passed handler.

func middleware(h http.Handler) http.Handler {
	// Prevents CSRF and Double Submits
	cs := csrfbanana.New(h, session.Store, session.Name)
	cs.FailureHandler(http.HandlerFunc(controller.InvalidToken))
	cs.ClearAfterUsage(true)
	cs.ExcludeRegexPaths([]string{"/static(.*)"})
	csrfbanana.TokenLength = 32
	csrfbanana.TokenName = "token"
	csrfbanana.SingleToken = false
	h = cs

	// Log every request
	h = logrequest.Handler(h)

	// Clear handler for Gorilla Context
	h = context.ClearHandler(h)

	return h
}

If you’re interested in reading more about the application, take a look at the README and code on GitHub: https://github.com/josephspurrier/gowebapp.

The next article is Go Web App Example - Views, Request Workflow, and View Plugins.

#go #code