This blog has been quiet for the past 4 months. And not only that, but most of my open-source projects have been quiet too. I just didn’t feel like working on it.

This is something that Adam Stacoviak and Jerod Santo talk about occasionally on their Changelog podcast¹, whose episodes generally involve interviewing people from the open-source community. I think they called it “project maintainer fatigue”, where an open-source project maintainer basically gets tired, at some point, with the work involved with managing their project.

I have very small projects, and I only dedicate a small amount of my free time to them, so I don’t get this a lot but it does happen every now and then. It usually doesn’t last this long though… I guess I’m getting old.

As is tradition, PieCrust 2.0 was released without much fanfare a few weeks ago. Just like with the previous version, it just happened because, well, nothing happened: I was using PieCrust for a couple other websites without any problem or need for new features, so I figured it might be as good a time as any to make it official.

Time to run pip install -U piecrust.

Since then of course I found a few bugs, so a 2.0.1 release is just around the corner, but that’s just business as usual.

What’s more interesting, first, is that I’ve been a bit quiet over here for the past couple months, and that’s because I was busy with my second hacking project from the holiday break (the first one of which was the recently announced Jouvence). Sadly it didn’t really pan out like I expected, so after working on it for a while I’ve now shelved it for the time being. These things happen.

The second thing is that I started work on PieCrust 3.0. It’s a mess right now but the goal there is to get rid of a bunch of cruft that accumulated over the years. As such, it will break backwards compatibility quite a lot (hence the major version bump) but I’m hopeful the result will be easier to use, easier to
maintain, and maybe even hopefully a bit faster.

I’ve been experimenting on and off with using Vim to write code at work, instead of Visual Studio, and recently I feel like I’ve reached a good enough spot to use it on a daily basis… here it is in action:

Keep reading if you want to know more!

Overview

Here’s the basic setup:

• CtrlP
• my fork of CtrlP Py Matcher
• Gutentags with either
  Exhuberant Ctags or Universal
  Ctags
• p44vim
• YouCompleteMe with clang support
• Some custom plugin (called fb.vim) that ties this all together specifically for Frostbite games codebases.

Vim Plugins

Listing Files

I’m using CtrlP to list files. This is your usual “fast open” thing, where you hit a keyboard shortcut, it opens a list of files, filters it down as you type a “fuzzy” pattern, and then gets out of your way. That’s what you see in the beginning of the video.

CtrlP requires a “root marker” to know what the root of a project is. However, Frostbite games have a lot of crap mixed up with all the source code – configuration scripts, tons of utilities, 3rd party libraries, etc. So asking CtrlP to index all of that would be madness.

The fb.vim plugin therefore generates a “lister script” that lists all the actual source files I care about – engine code, game code, etc. – and I feed that script as CtrlP’s ctrlp_user_command.

Note: Watch out, CtrlP requires passing the project root to user commands, even if you don’t need it because it’s already in some environment variables or something… so always add %s somewhere in ctrlp_user_command, otherwise you’ll get some completely silent errors that will leave you scratching your head for a while.

Pro tip: on Windows, you want to pass /-n /b /s /a-d to dir in order to get some nice output CtrlP can use.

Now you might be asking why I have a plugin generate the script, when I could have written the script directly by hand. Well if I was on a game team, working on the same codebase for 3 years, yeah maybe. But since I’m on the engine team, I have to juggle between dozens of branches, and dozens of different games – so I need the script to be portable, and generating it on Vim startup (when I detect the right environment) is the best way to keep that script up to date with my latest changes.

CtrlP Py-Matcher

Now the problem is performance. Even though I’m only interested in a subset of the codebase, that subset still has around 50k files in it… that’s where CtrlP Py-Matcher comes in.

The default search algorithm for CtrlP is implemented in VimScript so it’s, by definition, as portable as Vim, without the need to install anything. But it’s slow, whereas Py-Matcher is fast, but it’s implemented in Python so you need Python installed… but hey, good news, Python is installed anyways in a Frostbite environment! So yeah, that’s why I’m using that.

There are faster matchers out there for CtrlP (implemented in C/C++) but Windows binaries are hard to find, and Py-Matcher is good enough for this amount of files. Except that it had some Unix-isms that didn’t work well on Windows, so that’s why you may need my fork until my pull request is merged into the main project.

Tags

Ok now I can open files, but I can’t navigate the source code. Typically you need some kind of symbol database, which is where ctags comes in – either Universal or Exhuberant, pick your poison (see links above).

Gutentags (by yours truly) manages the tags file for you, so just install it and be happy. However, it also needs to be told what the root of the project is and, just like with CtrlP, indexing the entirety of the
root folder of a AAA game’s repository would be madness. So the fb.vim plugin (again) generates a “lister script” that will list the same files as for CtrlP, but for Gutentags.

Performance-wise, the resulting tags file contains around a million symbol definitions, and if I need to arbitarily jump to any of those through CtrlP, CtrlP Py-Matcher will take a second but will get there. Not super fast, but interactive enough.

So now I can navigate to any arbitrary tag, from anywhere. Yay!

Perforce

I can open files quickly, and navigate the source. But when I start typing, I need to potentially open that file for editing in Perforce, which is the source control management software that virtually all big game studios use. Unlike cool things like Mercurial or Git, all files in your repository are typically read-only until you explicitely check them out.

Fear not! Here comes P44Vim (by yours truly again), which is a super bare bones plugin for basic Perforce integration. It’s got only a couple commands, and automatically opens read-only file for edit when you start typing.

You don’t really see this part in the video since it happens behind the scenes.

Code Completion

Last but not least, I start typing some code, but there’s no code-completion! Usually I would go: “oh I need to give Intellisense another 10min before it wakes up”… but then I realize I’m not in Visual Studio anymore.

The biggest player in town for this is YouCompleteMe, which uses clang behind the scenes to compile stuff on the fly and give you almost-real-time feedback and completions. I was a bit frightened at first but they really tidied up their install procedure since last time I looked at it many years ago, and their
support for Windows seems pretty good. It only took me a couple minutes to get it going.

…except that, of course, we need to tell it how to compile our code. Once again, that’s where the fb.vim plugin comes in. It will go lurk inside the Visual Studio solutions and extract everything YCM and clang need to know – include directories, preprocessor defines, etc. – about the current file.

All of this happens behind the scenes, but that’s how you see the code completion when I call a method on the data variable, and how, when I later add an incorrect parameter to the function, it gives me feedback right away that this is wrong.

This has some issues, however. Our Windows build is not suposed to be compiled with clang, so I get a lot of warnings, along with a few errors, in the low level code (pragmas, attributes, SSE, etc.). I can pass the right flags to ignore the warnings (a whole bunch of -Wno-foobar), but not the errors. But that’s OK, since clang still builds a mostly correct AST for my current file’s compilation unit, and I still get mostly correct code completion and hints. It’s not perfect, but it’s already better than Intellisense, and faster/lighter than Visual Assist.

Future Work

It’s getting pretty cozy in Vim, here, but there are still a few things to do:

• Setup OmniSharp: I configured YouCompleteMe to give me all the goodness I need in C++, but haven’t looked into it for C# stuff yet.
• Invalid characters in our code: There are quite a lot of files with a UTF BOM header for some reason, and, more puzzling, a few with just invalid UTF8 characters in them. This makes python serialization very upset when Vim passes the tags file to CtrlP Py-Matcher, so I currently have a hacky post-process setup in Gutentags that filters out any symbol with such invalid characters.
• Debugging: I don’t think there’s any getting around Visual Studio for this, and I don’t think anybody would want to anyway since Visual Studio’s debugger is still the best around (don’t get me started on gcc), but I would like to experiment with running an empty Visual Studio (without the whole, super heavy, solution loaded) directly from Vim, with some commands to tell it to open the same file as I have in Vim so I can add breakpoints.

That’s All Folks

If you have feedback, suggestions, or comments about how much of an old bearded nerd I am, please share!

Over the holidays I started 2 hacking projects. This is the first one.

Jouvence is a Python library for parsing and rendering Fountain documents. If you don’t know Fountain, it’s basically Markdown but for writing screenplays. It was created by John August, Nima Yousefi, Stu Maschwitz, and a few other contributors – check out the official website for more information.

The code is, as usual, on both BitBucket and GitHub. The package is on PyPi. The API documentation is hosted on ReadTheDocs.

Here’s how the “Big Fish” screenplay renders with the default HTML renderer:

Of course the whole point of Jouvence is that you write tools and renderers of your own. The Jouvence parser returns a structured document object model that makes it easy to analyze, manipulate, or render screenplays any way you want.

For me, this let me add Fountain support to Wikked, my flat-file wiki engine.

Enjoy!

Happy new year and other such holiday things to the 3 people who read my blog!

I was in San-Francisco for new year’s eve and it was quite lovely.

Apart from the sake of posting something new, this post is also to show off how PieCrust’s admin panel can now upload page assets (in this case, that would be the picture above… check the URL, it’s hosted right here). This feature will be part of the impending 2.0 release, but you can get it by pulling the latest from BitBucket or GitHub as usual.

Cheers!

I’ve been critical of Bluetooth before. In my mind it’s nowhere near ready to be the default way we listen to music… but when my dear Bowers & Wilkins P5 headphones¹ started exhibiting balance problems, the nice people from the support department offered, among different options, to replace them with a discounted Wireless P5 model.

I figured what the hell, let’s try this brave new world where everything has to run on batteries and on different versions of different protocols.

Now you must realize that those are pretty much premium Bluetooth headphones. I got them at a 50% discount, but they otherwise sell for around $400. Even then, they’re much better that I expected. Barely heavier than the wired model, and with excellent battery life. I noted a few glitches in audio playback but I haven’t checked whether wireless was to blame, or whether it was the streaming or player app being at fault.

The thing that I was mostly wondering about was how it handles multiple devices. Our wireless speakers at home² have a lot of trouble figuring out what phone (between mine and my wife’s) to play from. My headphones would typically – on a daily basis – switch between my phone (for music and podcasts) and my trusty, original Nexus 7 tablet (which I use to run Plex and watch TV shows on the go). I sure didn’t want to go through all the trouble we typically have to go through with the speakers.

Well I can report that it works fine, but not in the way you would expect.

I have no idea how or why, but my headphones just won’t connect automatically to any of those devices. I have to connect to them manually³, which is frankly fine by me since it takes slightly less time than untangling a cable⁴ and plugging it, especially on Android where you can do a lot more system automation than on iOS (on my Nexus 7 it basically takes me a swipe and a tap, whereas on iOS I don’t think there’s a way around going in the Settings apps manually).

However, the simple fact that I have no idea if this is a feature or a bug should tell you all you need to know about the state of Bluetooth.

I’ve seen quite a few Apple bloggers link to this piece from Adam Geitgey about the new MacBook Pro, and how it’s supposedly “kind of great for hackers”. All because you can plug a lot of different things thanks to USB-C ports¹.

Huh… OK?

Well first, it’s great to see Apple users realize the benefits of standard ports – and based on how Adam seems to easily use lots of peripherals without any apparent problem, it’s good to know that some concerns about the compatibility landmine of USB-C may be overblown. I thought it was a given, that standard ports are better, and that yes, Apple ditching proprietary ports is an excellent (although baffling in some ways) development. But MacBook Pros having USB ports of any version is not a new thing. The only new thing is that they upgraded to the latest version of the standard… should we celebrate them for that? Isn’t that like saying “thank you” to cars stopping at red lights?

I feel like people are missing the point. Again.

The problem isn’t whether the new MacBook Pro is a good machine or not. It is a good machine. It’s the best MacBook Air that Apple ever released. It’s arguably an excellent MacBook, too. It’s just a shitty MacBook Pro because, in many ways, it’s a downgrade from the previous iterations – something that Apple does quite often.

But of course, it seems to pay off. Like I said before, Apple is now focused on mass consumer markets.

Someone (I don’t remember who) said that “Pro” now meant “Premium” in Apple’s line-ups. You don’t necessarily get a more complicated machine anymore. Instead, you get a fancier and pricier one. That’s arguably why they called the big iPad an “iPad Pro”. Not “iPad XL”, or “12” iPad”, or “the new new bigger iPad” or something. “Pro” is now the opposite of “Mini” and “Air”² – not the opposite of
“consumer” or “simple”.

I’m back from some travels – plural, which is extremely rare for me.

New York City (first time visit) followed by the usual annual trip to Stockholm for EA’s Frostbite DevDays conference, where various game devs from the company converge from all around the world to chat and drink.

EA is a weird company in the sense that, for a video game company, people tend to stay there for very long stretches of time. It’s very common to talk to people who have been at EA for more than 10 years – at EA Vancouver, DICE, Bioware, whatever. This makes it difficult to find new points of views on technical problems… although, well, maybe it’s the same in many other big companies like Activision or UbiSoft, I don’t know.

Either way, I was happy to meet several people who not only have shipped AAA games at other companies, but also worked on those games’ cameras – which is my current area of interest, being in charge of the Frostbite Camera System. Finding people who work on (and care about!) cameras is a challenge to begin with, seeing how little infrastructure and long term investments are generally done on that crucial aspect of any game (more on that in a future post), so I’m pretty happy with this year’s conference for that, at least.