A Windows C++ Ecosystem Without Visual Studio

published: 10 December 2016 • updated: 30 September 2019 • tags:

Windows is a hard environment for a C++ developer to live in – at least if you don’t use Visual Studio. What I find most annoying is setting up and maintaining several compiler toolchains in parallel. Compiler distributions – above all MinGW-w64 – are pretty much one-click installations these days, but easily switching the environment between different toolchains and moving those toolchains between computers ist still a hassle.

Since I’m lazy, I started scripting. Now I have a multi-toolchain ecosystem that can be moved between computers simply by copying a directory tree. Setting up and switching between environments is a single command. And even the first steps towards Qt Creator support are done.

That’s what this post is about.

I focus on MinGW with the GCC and Clang compilers. But it should be possible to integrate integrate Microsoft’s Visual C++ compiler as well, though I’ve never tried in earnest.

The Goals

Easy switching between multiple installed toolchains: Imagine you’re working on several projects. And imagine you need native Windows threads for some of these projects and POSIX threads for others. 32bit is probably still needed in addition to 64bit. And testing with different compiler versions is not unheard of either.

Now, how many toolchains is that? Even if you can have 64bit and 32bit combined, manually handling the environments for all of them becomes tedious really, really fast.

If all the toolchains are installed in a consistent way as shown below, setting up an environment for any of them and switching back and forth is completely automated. Also the switching mechanism – a bunch of Batch and Bash scripts – recognizes new and removed toolchains without manual intervention.
Support for both the native Windows Cmd shell and MSys Bash: Native is good. I’ve never liked faking a Linux-like environment for development purposes on a Windows machine. Making the toolchains run smoothly in the native Windows Cmd shell is a must-have.

However, especially when working with free (as in GNU “free”) and open source software that is not always possible. And this is where MSys comes in. MSys provides a Unix-on-Windows console environment with a Bash shell an all the essential support tools and infrastructure. Sooner or later you will run into the GNU autotools build system, and then a fully native Windows environment just won’t work anymore.

Of cource toolchain management for MSys has to be as automated as for Cmd.
Easily portable between computers: This point is less important than the other two. But still, it’s nice to have the complete toolchain ecosystem – compilers, utilites, source code, self-built binaries and configuration data – self-contained in one single folder structure. That way you can just copy the complete folder structure from one computer to another and keep working there immediately.

The Folder Structure

A fully configured and useable toolchain consists of three parts:

a system, i.e. a compiler distribution, probably MinGW.
a development set, i.e. your self-built executables, libraries and associated include files.
a bitness that applies to both the system and the development set. Can be either 64bit or 32bit.

Main Folders

This setup is reflected in the folder structure. Let’s take the example from above: MinGW GCC toolchains for Windows threads and POSIX threads; 64bit and 32bit each. Assuming the default base of C:\tc your folder structure would look something like the following:

Most of the folders have to be named as shown to work with the management scripts. Each one has its specific purpose to keep all the different parts of a toolchain neatly separated.

sys: A base folder similar to /usr on Linux. Here you install compiler distributions. In our example these are the four MinGW distributions.
<name>\win64 and <name>\win32: <name> stands for a compiler distribution. Keep the name short because you may have to type it to switch between compilers. The subfolders are for the 64bit and 32bit version of the same compiler. They must be called win32 and win64 and you cannot omit them. I.e. even if you intend to install the 64bit version only, you still need the win64 folder or the toolchain management scripts won’t find it. As an example, you would find the 64bit POSIX threaded GCC C++ compiler here: C:\tc\sys\gcc6-pthread\win64\bin\g++.exe.
dev: A base folder similar to /usr/local on Linux. Here you install different sets of your self-built binaries, libraries and associated include files. You can mix and match freely between compilers and dev sets as long as they are binary compatible.
<name>\win64 and <name>\win32: The same rules as for the sys subfolders apply. Additionally each of the win64 and win32 folders is supposed to contain at least the bin, lib and include subfolders. You’ll see the same in sys, so even though we’re completely Windows native a tiny bit of Linux peaks through from time to time – which is not entirely a bad thing.

The top level folder does not have to be called C:\tc. You can even rename it later and the ecosystem will continue to work. You might run into problems with some libraries, though. Qt, for example, is known to be really picky about its installation path.

Additional Folders

Just a quick overview of the other folders in the ecosystem tree.

cfg: Toolchain quick-switch configuration. Usually you switch a full toolchain by explicitly specifying all three parts: sys, dev and bitness. With config files in this folder you can give a short name to such a triple and switch to it that way.
home: Contains startup and switching hook scripts. These are your way to customize the toolchain setup.
msys2: A complete MSys2 installation; because sometimes your life needs a little bit of Bash – if you want it to, or not.
scripts: The management scripts (Cmd and Bash) for starting and switchting between toolchains.
sys\global: Contains toolchain independent software in the usual Unix-like subfolders (bin, lib, etc.). It’s a good place for things like CMake, pkgconfig or anything else you want to use for all installed toolchains.

Ecosystem Installation

Let’s install the ecosystem: this is what we’ll cover:

Version control: TortoiseHg, TortoiseGit
ecosystem core consisting mainly of the toolchain management scripts
additional tools: CMake, pkg-config
MinGW compiler toolchain
MSys2 as a Unix-on-Windows console environment
additional components: some MSys packages, a note about Qt

To keep things simple I assume C:\tc as the base folder where the development environment is located. However you can use almost any folder you like.

For all folder names it is essential that they do not contain spaces. Beware of the Program Files folder! Non-ASCII characters may or may not cause problems. So if at all possible keep it to letters (A–Z and a–z), numbers (0–9) and basic punctuation like the dot (.), dash (-) or underscore (_).

Version Control

TortoiseHg | Git for Windows | TortoiseGit

Mercurial (Hg) and Git are the two modern distributed version control systems you will not be able to avoid. Installing both with full Explorer integration is a good idea because it makes working with them really comfortable. However for setting up the development environment they are completely optional.

TortoiseHg is a self-contained package including the hg command line client for Mercurial and the TortoiseHg GUI components. Simply download and install and you’re ready to go. When installing the 64bit version make sure to enable the 32bit Explorer extension during setup. Otherwise the context menu extensions will not show up in 32bit programs.

For Git version control first install Git for Windows (keep the default settings) and then TortoiseGit because TortoiseGit does not contain a Git client itself. Once again when installing the 64bit version make sure to enable the 32bit explorer extension during setup.

Essential Components

Ecosystem Framework

Git repository

The framework consists mostly of the management scripts to setup a Windows Cmd or MSys console for a particular toolchain and switch between different toolchains.

To install it clone the repository to the empty C:\tc folder.

git clone https://gitlab.com/be-sc/hbb-win-cpp-ecosystem-framework.git C:\tc

Of course you can also clone with TortoiseGit or any other graphical Git frontend.

Some More Toolchain Tools

CMake | pkg-config lite

The following tools are a common requirement to build many software projects. Because they are not specific to one particular toolchain you put them into the sys\global folder. Every toolchain automatically has access to the everything installed there.

CMake is a popular cross-platform build system. It is used to prepare source code for compiling with Make, Ninja, etc. Download one of the Windows ZIP archives; it does not matter if you choose 63bit or 32bit. Unpack to C:\tc\sys\global ignoring the top-level folder in the archive. I.e. afterwards cmake.exe should be present in C:\tc\sys\global\bin.

Pkg-config is a tool for software (source code) library configuration management. Download the latest binary archive and unpack to C:\tc\sys\global ignoring the top-level folder in the archive. I.e. afterwards pkg-config.exe should be present in C:\tc\sys\global\bin.

MinGW

64bit downloads | 32bit downloads

MinGW packages the GCC compiler, the libstdc++ standard library and all the supporting infrastructure. After the installation it will be one of your compiler distributions in the sys folder.

The “bitness” of the download links refers to two things: the build architecture as well as the host/target architecture. I.e. the 64bit MinGW produces 64bit Windows binaries. The compiler and support tools are themselves 64bit Windows binaries. The same applies for 32bit.

Before you download you need to decide on a threading model and exception handling system according to the requirements of your project. If in doubt choose POSIX threads and SEH (64bit) or SJLJ (32bit) exception handling.

Unpack the downloaded 7-Zip archive to C:\tc\sys\<name>. You can use any toolchain name you like. But keep in mind that you have to type it when switching from one toolchain to another. I tend to use short names like gcc62 for GCC version 6.2. After unpacking C:\tc\sys\gcc62 contains a folder mingw64 or mingw32. Rename it to win64 or win32.

For the rest of the article I’ll assume a 64bit toolchain located in C:\tc\sys\gcc62\win64.

Now open a console and create the development folder structure where sources and self-built binaries will be located. Let’s call this dev set gcc.

cd /d C:\tc
md dev\gcc\win64
cd dev\gcc\win64
md bin
md include
md lib

MSys2

Installer | Project website

From the original MSys website:

MSYS is a collection of GNU utilities such as bash, make, gawk and grep to allow building of applications and programs which depend on traditionally UNIX tools to be present. It is intended to supplement MinGW and the deficiencies of the cmd shell.

MSys2 is an up-to-date and actively maintained version of the old MSys. For Windows software development its main purpose is to provide a Bash shell, which gives you access to the GNU autotools build system. Autotools relies primarily on Bash scripts for configuring the build and is still quite popular, especially in the open source world. If you’ve ever encountered a configure script you’ve had dealings with autotools.

Download the installer from the link above and install to C:\tc\msys2. The installer opens an MSys console for you. If it doesn’t, run C:\tc\msys2\msys2_shell.bat.

MSys contains the package manager from Arch Linux: Pacman. After the installation you need to update everything to the latest versions via Pacman. In the Msys console first update the core components:

pacman -Sy
pacman --needed -S bash pacman msys2-runtime

Then close Msys, run msys2\autorebase.bat (only if you are on 32bit), restart MSys via msys2_shell.bat and update the rest. Also install Make:

pacman -Su
pacman -S make

Now forget about msys2_shell.bat. You won’t need it anymore.

Ready to Use

From the scripts folder you can now start a native MinGW Cmd console with startmingw or MSys with startmsys. As long as you only have a single toolchain installed no further configuration is needed. The scripts will find the toolchain automatically.

The ecosystem comes with its own documentation. Open doc\index.html to learn more about the management scripts, the customization hooks they provide and toolchain config files.

Additional Components

Useful MSys Packages

Because of its package manager (Pacman) installing software into MSys is a simple task. Keep in mind, though, that MSys software might not work correctly when run from outside the Bash console.

The following software is a good addition to any development environment. Chances are high that you’ll need all of these at some point.

GNU Autotools

This build system is the main reason Bash is needed at all. When you download source code for the release version of a project usually a configure script created by autotools is included in the download. But that’s hardly ever the case when you get the source code directly from the project’s repository. Then you need to generate configure yourself. Autotools consists of three packages. Install them all.

pacman -S autoconf automake libtool

Subversion and CVS version control

Luckily CVS is mostly dead. But mostly does not mean completely. Subversion is still semi-relevant. It’s nice to have them available just in case.

pacman -S cvs svn

Patch

The classic tool to create a list of changes between two versions of a text file (called a diff) and apply them to files.

pacman -S patch

Qt

Website

Qt is a popular cross-platform application framework. In my opinion if you program in C++ and want to develop an application with a graphical user interface, Qt is hands-down the best option available.

Qt is huge and can take quite a bit of tinkering to build yourself. Unfortunately official binaries are only provided for MS Visual Studio and MinGW 32bit, but not for MinGW 64bit. So chances are high that you’ll have to build it yourself. That’s a topic for a another post, though.

Since Qt is such a monster in size I like to keep it separate. Accordingly the ecosystem has built-in support for installing Qt to dev\<name>\<bitness>\qt. The environment will automatically pick it up from there. Of course you can install it to just dev\<name>\<bitness> as well. Just don’t do both at the same time.