Making an extensible wiki system with Go

My little side project for roughly the last year has been a wiki system intended for enterprise use.

This certainly isn’t a new idea — off the top of my head, I can think of several of these “enterprise” wiki systems, with Confluence and Liferay being the most obvious (and widely used) examples.  The problem with the current solutions, at least in my mind, is that they have become so bloated and overloaded with features that they are difficult to use, difficult to administer, and difficult to extend.  I’ve been forced to use these (and other) collaboration/wiki systems and while I see the value in them, the sort of system I want to use just doesn’t seem to exist.

My goals were/are thus:

  1. Provide basic wiki functionality in a simple and clean UI as part of the core system
  2. Use markdown as the markup language (never liked wikicode) for editing
  3. Be horizontally scalable (I’ve suffered through overburdened enterprise software)
  4. Be extensible, both in the frontend UI (plugins) and the backend services
  5. Don’t run on the JVM (because reasons) 😉

Ok, that last one is kind of a joke (but not really). At least the core system ought to be native, while of course additional services can be written in just about any programming language.

After a year of working a few hours a week on this, I’ve come up with something I call Wikifeat (hey, the .com was available). It’s not finished, of course, and likely won’t be for a while yet.  Building an enterprise wiki/collaboration platform is proving a daunting task (especially goal #4 above), but the basics are done and the system is at least usable:

wikifeat_screenshot

Screenshot of the Wikifeat interface, showing a map plugin.

Technology

The Wikifeat backend consists of a CouchDB server(s) and several small (you might almost call them ‘micro’) services written in Go.  These services have names like ‘users’, ‘wikis’, etc. denoting their function.  Multiple instances of these services can be run across multiple machines in a network, in order to provide scalability.  The ‘frontend’ service acts a router for user requests from the Wikifeat web application, directing them to the appropriate service to fulfil each request.  Backend services  communicate directly with one another when needed as well via RESTful APIs.

The service ‘instances’ find each other via a service registry.  I decided on the excellent etcd to serve as my registry.  Each service instance maintains a cache of all of the other services registered with etcd that it can pull from when it needs to send a request to another service.

Extending the backend is a simple matter of developing a new service, registering it with etcd, and making use of the other services’ REST APIs.  The frontend service also has a facility for routing requests to these custom services (in the anticipated common use case of pairing a front-end javascript ‘plugin’ with a custom backend service).   Frontend plugins are written in Javascript and placed in a directory in the frontend service area.

The webapp itself is written as a single-page application with Backbone and Marionette.  The Wiki system is mostly complete.  Pages can be edited using CommonMark, a rather new implementation of Markdown.  I personally like Markdown for its simplicity, and always hated the various WYSIWYG HTML editors commonly included with CMS / Collaboration software.  Most developers already know markdown, and most non-techies should be able to pick it up quickly (being that it *is* meant to be a human-readable markup language):

wikifeat_edit

Wikifeat markdown editor

If that text editor looks familiar, it’s basically a tweaked version of the wmd editor used on stackexchange 🙂

Future Plans

I’m looking forward to continuing to evolve this thing.  The frontend probably needs the most help right now.  I actually hope to replace the wmd markdown editor with something more ‘custom’ that can make inserting plugins easier.  I’d also like to allow plugins to add custom ‘insertion’ helper buttons to the editor, along with a plugin ‘editor’ view, rather than requiring the user to enter a custom div block.

My mind is also overflowing with ideas for future plugins.  Calendars, blogs, integration with third party services/applications, etc.  Hopefully I can get to those eventually.  It would be *really* swell if someone else took on some of that work as well.  The project is open source (GPLv2 BSD), so that’s certainly possible…

UPDATE: After some feedback and reflection, I’ve decided to change the license from GPL to BSD 🙂