In the main article about the road to Diaspora, we looked at setting up our own pod to interact with the Diaspora federated community. Now we’re going to look at how that actually works. Or not. Because since I set up my pod a few weeks ago, I’ve had nothing but problems.
To give you an idea of how bad Diaspora is, even after a couple years of development, look no further than the bug tracker on their Github project page. After a week of using my pod, I had already posted half a dozen issues – and that’s for just one user on a lonely pod. I wasn’t following a lot of people either. Some of those issues have been closed, some are still open. But the point is: this is not good for a project that’s been in beta testing for so long.
Then, there’s the performance issues. Ruby on Rails has never had a very good reputation in this department, but I thought it was mostly a “haters gonna hate” kind of reputation, or some remnant of the framework’s early days, when it was not necessarily very optimized. But after running Diaspora, I’m revisiting my opinon. On an Ubuntu VM with 512Mb RAM, the thin server that runs Diaspora is the only process that ever got killed for running out of memory – neither WordPress or MediaWiki or anything else ran into this situation. And that happens on a regular basis. We’re talking a couple times a day on average. And don’t even think of running a Rails console at the same time: it may work for a while, but you better get done quickly because one of the 2 processes will die soon enough.
And then there’s the whole “federation” aspect, which is the whole point of Diaspora. Spoiler warning: it doesn’t work. I don’t know if it’s related to the performance problems above (the process could be dropping important bits of information when it gets killed) but I always seem to be missing posts and updates from anybody else that’s not on my pod. Doing some testing between my pod and a few other well-known pods like joindiaspora.com or diasp.org, I get completely random results: sometimes a comment or “like” gets propagated in a few seconds, sometimes it takes hours, and sometimes it just doesn’t show up at all on one end or another.
I could go on and on about all kinds of little problems, from the completely stupid “PersonName started sharing with you!” email that doesn’t make any sense (because it means they’re following you, which means they won’t share anything with you unless you follow them back) to some weird design decisions around hashtags (try figuring out how to follow hashtags from other pods) to some obvious problems that never seem to get fixed (like useless Diaspora links to your Twitter cross-posts, or the inability to cross-post public posts to Facebook). I guess that’s what the Github bug tracker is for but, again, this really doesn’t look like a project that’s more that 2 years old.
Diaspora’s future
The original Diaspora founders recently left the project, saying they’re “giving it back to the community” while they “take the back seat”, a.k.a.Makr.io, a very stupid website concept. Some people say that’s the death of Diaspora, but judging from the state of it, it may actually be what saves it. The project was obviously badly managed and designed from the start, so maybe, just maybe, having a community of better programmers take over the codebase would make it viable.
There’s also the very, very slim possibility of someone either re-implementing Diaspora using a different framework than Ruby on Rails. Diaspora is mostly based on open standards – although the keyword here is “mostly” – so it would be possible to rewrite something from scratch that’s compatible.
There’s also the very, very slim possibility of someone writing something else that works better. Mike Macgirvin, the creator of Friendica, recently started working on “Project Red”, which is “Friendica taken to the next level”. His announcement didn’t hesitate to make fun of Diaspora:
Friendica WORKS today (unlike similar projects which are still struggling at basic communications after two years, and after squandering huge amounts of money).
Seeing how Friendica works and is easily installed – even though it’s ugly and useless to me – Project Red could be a good thing if it’s based on asymmetric relationships and Mike can enlist the help of a moderately talented web designer.
In the meantime, I don’t think we’ll get people away from Twitter and Facebook any time soon.
The first step in the journey to Diaspora is to get your own Diaspora server because, well, that’s the whole point of a distributed social network: you get to own your stuff (you could argue that, on the other hand, I’m not running my own email server, but, err, whatever, indulge me).
Unfortunately, setting up a Diaspora pod is insanely convoluted and complex.
After the jump, we’ll get into the meat of things and hopefully it will help you with the process (if you ever want to attempt it).
Looking at the complexity of setting up a Diaspora pod, there’s something to be said about the Ruby community, who often seems to degenerate into a pile of over-engineered, over-complex set of weirdly named libraries and tools, all of which end up being very slow and memory hungry. Hipsters was the worse thing that ever happened to the Ruby language.
Anyway, enough trolling, you probably want to start by following the installation guides, over at Github. My server is an Ubuntu virtual machine so I followed the Ubuntu guide which will make you apt-get install a whole lot of packages, from system libraries to Ruby and the usual suspects (gem, bundler, rake, etc.). Once that’s done, you move on to the Diaspora installation guide itself, which is where things get fun (given a certain definition of “fun”), and where I’ll try to hold your hand (no homo).
Note that several “easy installer” initiatives are being worked on, mostly targeted at popular hosting companies like Heroku. By the time you read this, you may have something available for your situation.
Get an SSL certificate
The first thing you need is an SSL certificate for your domain. Since you’ll be exchanging potentially private communications between your pod and the other pods, it’s a good idea to go through https, which means getting an SSL certificate. You could usually use a self-signed certificate for this, but the bigger pods (a.k.a “community pods”) won’t accept that, so you need a proper certificate.
The documentation will point you at StartSSL, which delivers free class 1 certificates for your domain for a year. That’s enough to get you started. Just go to their website and follow the instructions – you’ll end up with a few keys and other serious-looking files. If you plan on hosting your pod on a sub-domain, don’t forget to add your top-level domain to the alternative domains falling under the certificate.
Getting Diaspora
If you keep following the installation guide, you will at that point have cloned the Diaspora repository on your server, run a few commands like bundle install, and edited a few configuration files. Here are a few tips, though:
Before you do all the RVM and bundler stuff, make sure you apt-get install libreadline-dev libncruses-dev. Hopefully this will make the readline gem work right away. We’ll talk about that later.
For some reason, the config/application.yml.example file features a pod_url with a port 3000 specified. Make sure you take this out when you replace it with your own domain URL. This has to be the real, public URL (otherwise your Diaspora ID will have :3000 in it!).
For me, the ca_file was /etc/ssl/certs/ca-certificates.crt, but it of course depends on your Linux distribution.
Of course, you’ll need to follow the instructions for setting up a “production” server. Don’t set serve_static_assets to true, there’s no need for that – Apache will serve the static assets already. Set single_process_mode to false, too, because it looks like it’s mostly for development/debugging purposes.
I’d recommend setting up the mailer right away (it’s the thing that sends notification emails). For some reason, even when mailer_on is false, Diaspora will try to send some emails, resulting in errors and failed jobs. It’s not a huge deal (maybe), but I like to avoid false positives in my error logs. Here it goes:
Install sendmail with some apt-get or something.
Set the mailer_method to sendmail.
That’s it! (although you can optionally set some nice email address for the sender).
Setting up the database
For some reason, the config files assume you’re going to run the app with the root MySQL user. I don’t know what they’re smoking over at DiasporaHQ but there’s no way I’m going to do that, so instead I created a new database myself, called diaspora_production, and a user with access to that database. You could call the database differently, I guess, but you would have to go and change the default name in all the configuration files. And I’m not sure if there’s any hard-coded stuff somewhere that references that database, so I went with the safe route of using their naming convention.
Once that’s done, you don’t need to run the db:create Rake task. Just run bundle exec rake db:migrate. You should see your database filling up with tables.
Apache configuration
Now you’ll need a web server to actually get anything working. I’m using Apache for this.
The first thing is to make sure the SSL/https stuff will be working:
If you don’t have that already, add NameVirtualHost *:443 and Listen 443 to your conf files to make Apache listen to incoming connections on port 443 (which is used for https). You can put that next to the similar directives listening to port 80 (which is standard http).
You’ll need to open port 443, obviously, in case you have a firewall (which you should have). Again, just look for where it deals with port 80, and copy/paste. If you’re using iptables, you should end up with something like -A INPUT -p tcp --dport 443 -j ACCEPT.
Make sure you have mod_ssl loaded/enabled.
Then, add an entry for the new website (your pod). You can use this Apache configuration that they point to in the installation guide. The only modification I did is to add a reference to the SSLCACertificateFile I got from StartSSL. So all my SSL stuff would look like this at the end:
SSLEngine On
# You generated those with StartSSL
SSLCertificateFile /path/to/cert.crt
SSLCertificateKeyFile /path/to/private_key.key
# You got those from StartSSL
SSLCertificateChainFile /path/to/sub.class1.server.ca.pem
SSLCACertificateFile /path/to/ca.pem
For the rest, you don’t need to change much except the path to the Diaspora repository you cloned with Git. Make sure you specify the path to the public folder, not the root folder, otherwise it won’t find all the CSS and images and such.
Also note that you’ll need mod_proxy, mod_proxy_balancer and mod_proxy_http loaded/enabled for this to work.
Running the pod
Holy shit. See, what did I tell you, that it was insanely complicated? Well, it’s not quite over yet.
Now you can at last run ./script/server, wait a bit, and go visit your brand new pod. If everything’s good, you should have a little “lock” icon in your address bar (and if you click on it, it tells you your domain is verified by StartCom Ltd.). You should also see the Diaspora welcome page.
Go and signup! It’s all yours!
When that’s done, you can go back to config/application.yml and set registrations_closed to true so that nobody else can create an account (if that’s what you want).
Kill the server with CTRL+C and, while it’s down, open the Rails console with bundle exec rails c.
Remember when I talked about the readline stuff? Well, if you didn’t do it well, that’s where it fails. If so, you’ll have to patch the readline gem like I had to.
Go to ~/.rvm/src/ruby-1.9.3-p125/ext/readline and then ruby extconf.rb, make, and make install. Make sure you’re using the same version of Ruby to run extconf.rb than the version you’re patching! (if you only have one Ruby installed, you don’t have to worry about that last detail).
Running the console is for adding your newly created user to the “administrators” role, so you can get access to the administration dashboard: go to this FAQ page and look for “Roles”. You should see a bunch of stuff you need to type in the console to make yourself an administrator.
When you’re done, start the server again with ./scripts/server.
Administration tips
On your pod’s website, in the top-right menu (where you have “Contacts” and “Log Out”) you should see a new entry called “Admin”. Click on it. The most useful page here is the last one: “Resque Overview”.
Resque is the background job manager, which takes care of a lot of important stuff for your pod. The overview page will show you, among other things, any failed jobs. This is useful to troubleshoot problems and send bug reports.
Another hidden page is at /admin_panel. This is the Rails panel dashboard, where you can basically hack into your database to fix things, compare values, or clean data. It’s a bit dangerous but can be useful at times.
Conclusion
That’s it! If you made it this far, congratulations, you have absolutely no life. You should, at this point, be able to send useless status updates to your so-called friends, with the warm feeling that comes with using open standards.
In the next post in this series, we’ll look at all the problems you will face with your Diaspora pod – and there are many, unfortunately.
Recently, app.net has gotten a lot of attention, but I just don’t see the appeal. It’s basically a Twitter clone that you have to pay, and all of this for what? So that the API is nicer to developers and you don’t see a couple of “promoted tweets” once in a while?
It sounds like a very shallow goal for a supposedly “disruptive” communication platform. Sure it has some kind of grand plan to get us to the next level of connectedness through, err, innovative apps and mashups or something. But it doesn’t make things better on the ownership level. It’s still yet another data silo. And I’m fed up with silos.
Remember when we used to communicate through free, open, distributed and standardized protocols? You know, like emails or phone calls? Or even snail mail? My problem with Facebook or Twitter is not that I’m the product, but that I don’t own my data, and that there’s no competition or choice between service providers. They’re not only data silos, but business model silos.
Other people also got fed up with the slow decline of the open and connected web, and started writing the building blocks for distributed social networks. Several projects emerged from this, with the most famous being Friendica and Diaspora.
Friendica: its main advantage is that it’s easy to install – like most PHP projects, you just copy files on your server, edit a configuration file, and follow the instructions on the web. You just have a couple of additional steps after that to setup your crontab so it can synchronize posts in the background. It also has very good interoperability with pretty much all the other social networks. The big problem, however, is that it’s batshit ugly. Also, as far as I can tell, it only allows bi-directional relationships, like Facebook, which means that if you want to “follow” somebody, you’ll have to send a “connection request”.
Diaspora: announced with much fanfare a while ago, it felt like vaporware for the longest time until it actually looked like they were shipping something. The selling point is a Google+-like “friendship” model, where relationships are asymmetrical and can be classified with “aspects” (Diaspora’s version of Google+’s circles). However, as we’ll see in the next post in this series, it’s insanely complex to install, full of bugs, and has poor performance. It also only allows to cross-post to select competitors (Facebook, Twitter and Tumblr).
With that in mind, Diaspora doesn’t seem to me like the best alternative to commercial social networks – it’s the only one so far. In this journey to Diaspora, we’ll look at:
Almost 4 years ago, I wrote a short article on dumb websites who have a maxiumum password length.
Now, in 2012, there are still websites with such stupid policies. One of the most famous is none other than Microsoft’s Live Account service, which serves as the authentication hub for all things Microsoft. Basically, your Live ID, or whatever it’s called, can’t have a password longer than 16 characters.
Microsoft is, rightly so, getting a lot of criticism about that because the recently released Windows 8 lets you link your Windows user to a Live ID, for use with the whole Windows App Store thing and more. This means, in most cases, that your Windows password can’t be longer than 16 characters, and, in other cases, that you have all kinds of weird account related bugs and you may need to enter a truncated password.
Back in the first part of this 2-part post we looked in some detail at how MacOS mounts network shares, and how badly designed this feature is compared to its Windows counterpart.
We’ll now look at the solution I’m using to fix the problem, which is to mount network shares in a consistent way for a multi-user machine.
Thanks to the power of UNIX, you can get around the problem by spending hours reading boring documentation, searching useless forums, editing obscure configuration files and generally speaking wasting your time for something Windows gets right in 2 clicks.
If you do the research, you will find lots and lots of solutions – some clever and some completely stupid. Not all of them worked for me, so here’s the solution I came up with. It uses the autofs feature supported by most recent versions of MacOS, and has some pros and cons. “Your mileage may vary”, as they say.
Creating custom mount points
Start by editing /etc/auto_master. It should look like the following, but without that last line. Add it using your favorite text editor (which you may have to run as root):
This file tells the automounter to treat the folders specified at the beginning of each line as mount points for the network shares specified at the end of each line. The file protocol I’m using is smbfs, which is the Windows file-protocol. You could try to use afp here, which is the Apple File Protocol, but I ran into permission problems on files I was editing or creating on my server. Also, depending on your server, smbfs may be faster.
I’m mounting my network shares in each user’s home directory, in a ~/Volumes folder, but you can mount them anywhere else – the whole point here is to have one “mount sandbox” per user instead of the system-wide /Volumes.
Also, here I’m specifying network shares with a custom username and password. You may not need that if you’re using the same credentials on both the local machine and the server. Of course, replace server with your actual server name or IP.
Once that’s done, run sudo automount -vc to tell the automounter to refresh. If it doesn’t work, you may have to create the ~/Volumes directory, along with the mount points themselves. I also ran into problems where the automounter would still be a bit confused and had to reboot to make things work, so you can try that as a last resort.
Accessing the network shares
That’s the totally lame part: the mount points you created are not displayed in Finder until they are actually mounted. They will show up only after they’re mounted, which only happens when needed, i.e the first time they are accessed. So, at first, your ~/Volumes directory will be empty – at least in the Finder (if you go there using a Terminal, you will see the mount points, and you can cd into them, which would trigger the mount and make them appear in Finder! Yeah, it’s confusing, I know, welcome to the Apple ecosystem).
How do you access your network shares if you can’t see them in the Finder, then? Well you go directly to them with “Go > Go to Folder…” and enter, say, ~/Volumes/media:
You would only have to do this once in a while because once they are mounted, you’re unlikely to unmount them (you would actually need to use the command line for that). Also, once you gave that path to an application like iTunes or Lightroom, those applications can trigger the mount themselves by merely accessing the path: the next time you reboot and launch the application, it will all magically work (although you may notice a pause for a second or two as the system mounts the network share).
Alternatives
If you want the network shares to be visible in Finder as soon as you login, you can either:
Make a login item (a Bash or Apple script, for example) that will trigger the mount by accessing the path somehow.
Specify mount points in /etc/fstab instead of through the automounter.
I myself prefer staying with the automounter, because it means I won’t get a timeout when I log into my laptop but I’m not at home. Since I rarely reboot, it’s very uncommon for me to have to manually re-mount my network shares.
Conclusion
This is what makes it possible for me to use my home server along with sharing my Macbook with other users. It was a lot more complicated than I anticipated, but I guess making Macbooks play nice with custom servers would go against Apple’s new cash cow, a.k.a “iCloud”.
If you asked me a year ago what was the most awesome feature that Windows has and that MacOS doesn’t, I would have probably scratched my head for a bit, mentally sorting through all the obscure advanced things you can do with the Windows SDK and a few lines of code, or all the little things that make organizing files so much easier than with the horrible Mac Finder.
But if you ask me now, I’ll reply straight away this: mapped network drives.
You would think there’s not much to it, but this has been my biggest problem as a user when I switched to a Macbook Pro as my main machine. In the first part of this 2-part post, we’ll look at the problem. Go to part 2 for the solution I’m using.
In case you don’t know, here’s a little crash course in mapped drives. You can skip ahead if you’ve already used them.
Windows famously uses drive letters for each physical or logical drive hooked up to your computer. This means in most cases you would access your main system drive as C:, some secondary drive as D:, a USB thumbdrive as E:, etc. However, you can also be part of a network, and this means you would access a server’s shared directory as \MyServersharename.
To make things simpler, you can map a server’s shared directory to a new drive letter like Z:. This is not an advanced feature. In fact, it’s always right there in the explorer’s toolbar when you open “My Computer”.
By the way, you can see here my typical setup at home, where I mount 3 shares from my file server: one for my personal data, one for the family data like pictures and home videos, and one for miscellaneous media like movies or music.
Setting up a mapped network drive takes about 2 clicks and a few keystrokes:
Click on “Map Network Drive”.
Type your server’s name and share name.
Optionally enter specific credentials if the share is protected.
Done!
At this point, you have a simple and convenient access to your server in a way that’s totally transparent to any application you may run. You can for example run a media manager like iTunes or MediaMonkey and tell it to look for music in Z:Music. It will always work – at least as long as you checked the “Reconnect at logon” option, and you’re of course connected to your network.
Doesn’t MacOS have something similar?
It does. Kinda.
When you’re connected to your network, you will see the available servers in the “Shared” sidebar of the Mac Finder:
My server here shows up multiple times because it exposes several file protocols that MacOS understands (comparatively, out of the box, Windows only understands its own file protocol and not the Unix or Apple ones).
Once you click on a server, you can browse its shared directories that are available to you, and when you dive into one, it will “mount” it on our file-system. You can tell it’s mounted because the Mac Finder will show an “eject” button next to each mounted share, and next to the server itself. Here I mounted 2 shares:
Just like on Windows, the server share is mounted in a way that should be transparent to applications. So it should all work out fine, right?
Well, no.
Under the hood
MacOS is based on a UNIX-like architecture, so there are no drive letters involved here – only mount points that look like directories on the local file-system.
Sure enough, if you open a Terminal and go look into /Volumes, you’ll see the mount points for the stuff I mounted in the previous screenshot:
My media folder on my file server is mounted as /Volumes/media, and if I point an application like iTunes to it, that’s the actual path it will use behind the scenes.
The problem of multiples
What happens if another user (say, my wife) logs in and also wants to mount the media folder to listen to some music? Or what happens if you want to mount another share called media from another server?
Well, MacOS, just like any other UNIX-based system out there, does something very stupid: it appends a number at the end of the mount point:
My wife’s media folder is, under the hood, mounted as Volumes/media-1. Other attempts at mounting something with that name will result in Volumes/media-2, Volumes/media-3 and so on.
Of course, this all depends on who mounts things first. If I was the one who logged on to the Macbook later, I would have been assigned media-1 when my wife would have media.
Compare that to Windows’ mapped network drives, which are user-specific, which means there’s no problem as long as you assign the same drive letter to the same network shares – which is a trivial thing to do.
Bad things happen
This is where things break down rapidly.
Did you originally, and unwillingly, tell iTunes your music was in /Volumes/media? Well, now your whole library is empty because it can’t access that directory – it either belongs to one of the other users, or it has been unmounted. Worse, it could actually point to a completely different share that you mounted from another server because it has the same name.
The same thing happens with your Lightroom pictures, your documents or presentations with embedded assets, and any other program that stores paths to some type of data you may have on a file server. All of a sudden, life is not worth living anymore.
The cynical in me tells me Apple doesn’t care about you if you’re sharing your laptop with someone else anyway – they’re not interested in cheap people – but it could actually happen even if you’re the only user: if there was a problem with the server, or with the network connection, well, let’s just say MacOS will be very happy to give up trying to mount your network share at its original mount point, and will instead do it with a new mount point, again with the number suffix. So you could end up with /Volumes/media-1 even if you’re the only user logged in.
The 0.8.0 (and even 0.8.1!) version of PieCrust has been tagged in the stable branch.
As usual:
I still need to write some documentation on the new and/or changed features.
I’m really not good at keeping a single release focused around a small set of consistent new features. I tend to pack different unrelated features mixed with bug fixes as they come to me, and the result is a bit messy.
You can read about the changes in the CHANGELOG, or keep reading for a detailed description of the highlights. Or you can just go and grab it from BitBucket or Github and trust me that it’s awesome! (but wait, you should at least read the first couple sections here below because there are a few breaking changes).
As we get closer to a 1.0 release, I changed the folder structure to make it look more like a genuine application. It used to look like an already installed system (a _piecrust folder with the code and a sample website folder), which was nice during early prototyping and development. Now, however, it doesn’t feel right anymore, especially since chef has evolved into a sophisticated tool.
So the folder structure now has a bin folder with chef, the usual src, libs, tests folders, and a single piecrust.php file to include at the root.
For backwards compatibility purposes, the important files are also available at their old location (chef and piecrust.php) but they will issue a warning message if you use them from there. They will be removed when we hit version 1.0.
Broken stuff
There was a few things that didn’t quite make sense that I also fixed, unfortunately resulting in breaking changes:
When pretty_urls are disabled, pages with pagination will have the same first page URL as if they didn’t have sub-pages (e.g.foo/bar.html). Sub-pages will have URLs like foo/bar/2.html. This makes URLs more consistent.
PieCrust now supports extensions other than .html for pages. Before, you had to make all files in _content/pages with an .html extension, and use the content_type configuration setting in the header to change the output to something else, like .xml or .rss or whatever.
Now, you can actually create files like _content/pages/feed.xml and it will be handled correctly. This means content_type is now only meant for setting HTML response headers in CMS mode, which makes more sense.
Some chef commands had some inconsistently named options: some would have hyphens, some would have underscores, and some would just concatenate words together. I don’t know why I never noticed it until now… In most cases, the old option name is still available, but will issue a warning that it will be removed when we hit version 1.0.
Multiple formats
Thanks to the magic of open-source, PieCrust now also supports changing the format of a page right in the middle of the content. This is useful when you’re using a minimalist format like Markdown and just need that extra feature at one point to control some CSS or HTML property without having to write it all verbatim:
So here I am, writing some stuff. It's all _nice_ and _fun_.
But then, I need to write a boxed piece of text that will be laid out nicely with CSS. Let's use HAML!
<--haml-->
.boxed_text#tutorial1
.left.column
:markdown
This is some **cool stuff** right here!
.right.column
:markdown
Hey, this is nice too!
<--markdown-->
Ok, back to normal, now. **Whew!**.
This example obviously uses the PhamlP plugin (which adds the Haml and Sass languages), but you should get the idea.
Also, if you want to start a content segment using a different format than the one defined for the page, you can append the name of the format you want to use like this:
---newsegment:textile---
Another content segment using **Textile** formatting!
Templating features
The iterators obtained through pagination, blog.posts or link have a few new tricks:
You can sort posts and pages with a new .sortBy(foo) function. Here, foo is the name of a configuration setting in the pages’ or posts’ headers that will be used to order those pages or posts.
You can now access the content segments of pages and posts you’re iterating on – not just metadata like title and date.
You can use paginagion.all_page_numbers to get, well, all the page numbers for the current page. You can also use pagination.page(i) (where i is a number) to get the link to a specific page. This will help you build pagination footers more easily.
Portable baking
The --fileurls option to chef bake used to bake a website with absolute local paths for all links created with the pcurl family of template functions. This made it easy to just bake a site and double-click on the main page to preview it locally without any web server.
It was also flawed, because it meant the site could only be previewed in that exact place on the file-system – you couldn’t give it to a colleague or client to review.
A new option, --portable, effectively replaces the --fileurls option (which is now deprecated). It will create relative links (with lots of ../ in them) so that you can move the baked static files around and still have them previewable locally in a browser.
Debug info in CMS mode
For people running PieCrust in dynamic CMS mode, you can set site/enable_debug_info to false in the site configuration to disable the ?!debug feature, which could potentially expose private information to visitors.
Recently I realized I’ve been working on, and talking about, PieCrust for quite a while, but I have never given any reason as to why on Earth I have written my own CMS and static site generator when there are so many already out there.
Like many open-source projects, PieCrust was born out of 2 very self-centered stimuli.
And by “new”, here, I’m of course talking about things that were new to me at the time. Writing PieCrust, I learned a lot about web technologies, from HTTP servers to caching and proxies and whatnot.
With my next projects, I’ll learn a lot more – I’m looking at databases and scalability and security and other bigger, more complex things – but it was good to start with something simple and basic.
Two: everything else sucks!
Well, that’s not true, but you know what I mean. The average developer (and that includes me) is usually never happy with what’s available out there.
Most other static site generators are completely OK – they work well for the most part – but they have 2 flaws: they’re simplistic, and they’re doing it wrong.
Simplicity
Most other engines are too simplistic. They confuse simplicity with a lack of features. I believe you can have a fairly fully fledged static site generator without sacrificing how simple it is to use and work with.
Most other engines are built from the ground up to be static site generators. This means that when it’s time to preview your work in a browser, what they do is just re-generate the whole site and show the resulting HTML file matching the request. Thankfully, they do it incrementally, like a compiler, which means they only generate what was changed, but it’s not ideal unless you’re just looking at simple pages.
If you’re modifying a page with a lot of dependencies, that re-generation can take several seconds, which means you have to wait until you see your updated work. This happens, for example, when you’re working on a blog post that has various tags and categories: updating it means not only re-generating that post’s page, but also the home and/or archive page on which it appears, and all the relevant tag/category listing pages. Some static site generators take a few shortcuts here by not re-generating the whole site in that case, resulting in glitches in the preview.
This “compiler-like” approach works well when you always want the full result of the generation, but in the case of a user working on some content for his website, he only wants to see one page. You can only press F5 on one window at a time. So whatever dependencies a piece of content has, the engine should be able to just generate what the user wants to see. And this is what CMSes have been doing for a long time.
That’s why PieCrust is built from the ground up as a database-less CMS. Sure, it can generate a static site, but that’s just a side-feature. The focus is on what the user does the most (iterating between writing and previewing), and not what the user wants (a static website). What he wants happens later, once he’s done working.
The nice thing is that PieCrust ended up being both a good CMS (especially if you use it with a reverse proxy) and good static site generator (I’ll post some comparative benchmarks someday).
The new version of PieCrust has been tagged in the stable branch yesterday, and oh boy does it have some cool features. You can read about them in the CHANGELOG, or keep on reading for a more detailed presentation.
Until now, if you wanted to display a list of posts on your website, you had to go through the pagination.posts variable, which has the nasty side-effect of generating pagination for the current page (i.e. the page would have as many sub-pages as needed to show all your posts, with some expected links like “next entries” and “previous entries” to navigate between them). You could of course use the single_page setting to prevent any sub-pages from being created, but that was a bit awkward, especially since you also probably wanted to tweak other things, like the number of posts to show.
Now there’s a more straightforward way with the blog.posts variable, which returns all your posts without any limit or filtering. It also doesn’t affect pagination in any way. The nice thing is that you can still skip, limit or otherwise filter the posts.
For example, on my own home page I show the latest 7 posts. This can be done like this:
{% for post in blog.posts.limit(7) %}
* [{{ post.title }}]({{ post.url }})
{% endfor %}
You can also use Twig’s built-in slice filter to skip and limit the posts, as in:
{% for post in blog.posts|slice(0, 7) %}
* [{{ post.title }}]({{ post.url }})
{% endfor %}
But you can also do more advanced things, like:
---
my_filter:
has_tags: cooking
not:
is_status: draft
not:
has_tags: appetizer
---
{% for post in blog.posts.filter('my_filter').limit(5) %}
* [{{ post.title }}]({{ post.url }})
{% endfor %}
The blog variable also contains other cool things, like tags and categories, and, because this whole feature is called “blog archives” after all, years and months. This makes it possible to do the following:
# Blog Archives
{% for month in blog.months %}
## {{ month.timestamp|date('M Y') }}
{% for post in month.post %}
* [{{ post.title }}]({{ post.url }})
{% endfor %}
{% endfor %}
This is pretty cool: PieCrust can now chain your file processors when baking (and even previewing!) your website.
This means that if you have, say, the LessCSS and YUICompressor processors (the first one ships with PieCrust, the second one is available as a plugin), not only will your Less files be compiled to CSS, but they will also be further processed by the compressor, resulting in a compiled and compressed file!
The pagination object now has two new properties: next_post and prev_post.
These objects will be valid if the current page is itself a post, and they will point to (unsurprisingly) the next and previous posts in the blog. Those are the full blown post objects, which means you have access to the title, timestamp, etc.
This is useful if you want a blog layout where you have links to those previous/next posts at the bottom of each article (like this very blog!).
New chef commands
Chef, the command line tool for PieCrust, got a few new commands of its own:
chef showconfig prints parts or all of the website’s configuration settings.
chef find lets you find pages, posts and templates in the current website.
chef plugins now has sub-commands to let you search and install plugins from a plugin source. Right now, the only type of plugin source that works is a BitBucket source (it will look for any repository that starts with PieCrust-Plugin-), and the only default source is my own BitBucket account (you can add more sources with the site/plugins_sources configuration setting).
New importers
You can now import posts and pages from a Jekyll or Joomla website.
Also, note that the chef import command changed:
The format and source options are now mandatory arguments – although if you use the old syntax you will be warned about the change.
You can now specify importer-specific options, like the WordPress SQL table prefix, as a command-line option, which is a lot better than the previous awkward connection-string suffix hack.
Twig extensions
The built-in Twig extension now comes with the additional following features:
A striptag filter, which strips outer HTML tags from a string.
A textfrom function, which includes text from an arbitrary text file. This is useful if, like me, you write your blog posts as simple text files stored in Dropbox, but you sometimes want to preview the article in the context of your blog without having to copy/paste the text.
Also, you can set the twig/auto_escape site configuration setting to false if you don’t want Twig to auto-escape your stuff automatically. This means you won’t have to pass all your articles through the raw filter in your templates anymore, but it means you will have to make sure you escape other stuff accordingly.
Miscellaneous changes
Pages are now cached differently: they used to be parsed, formatted and rendered in one go, with the final result cached to make it super fast to re-use it templates. However, it broke some advanced use-cases with Twig and other template engines that support inclusion and such things.
Now, only the parsed pages are cached – formatting and rendering will happen every time the page is requestedduring a session. This actually doesn’t affect performance that much because, within a session (a preview request, or a baking), the final page is cached in memory anyway.
The bake information file (bakeinfo.json) used to pollute the output directory. Not anymore, as it’s now saved in the cache directory.
The Less and Markdown libraries have been updated to their latest version.
That’s it! Go grab PieCrust 0.7 from Github or BitBucket as ususal, and be sure to send me feedback and bug reports.
