Lab7 : Build system - Qualité de Développement

The goal of this Lab is to understand the following points:

Structuring your project¶

Here is the typical structure of your project:

Lab7/
└── main.cpp

main.cpp

#include <iostream>
#include <string>
#include <vector>

class Book {
private:
    std::string mTitle;
    std::string mAuthor;

public:
    Book(const std::string& title, const std::string& author) : mTitle(title), mAuthor(author) {}

    std::string getTitle() const {
        return mTitle;
    }

    std::string getAuthor() const {
        return mAuthor;
    }

    void display() const {
        std::cout << mTitle << " by " << mAuthor << std::endl;
    }
};

class Library {
private:
    std::string mName;
    std::vector<Book> mBooks;

public:
    Library(const std::string& name) : mName(name) {}

    void addBook(const Book& book) {
        mBooks.push_back(book);
    }

    void displayBooks() const {
        std::cout << "Library: " << mName << std::endl;
        for (const Book& book : mBooks) {
            book.display();
        }
    }
};

int main() {
    Book gameOfThrones("Game of Thrones", "G. R. R. Martin");
    Book theHobbit("The Hobbit", "J. R. R. Tolkien");

    Library library("City Library");
    library.addBook(gameOfThrones);
    library.addBook(theHobbit);

    library.displayBooks();

    return 0;
}

Compile and run the program. Observe the output.
Clean the project by removing files generated during compilation.
Restructure your project as seen in class using the following structure and makefile:

Lab7/
├── include/
│    ├── book.h
│    └── library.h
├── source/
│    ├── book.cpp
│    ├── library.cpp
│    └── main.cpp
└── makefile

makefile

all: library-app

library-app: source/book.o source/library.o source/main.o
	g++ source/book.o source/library.o source/main.o -o library-app

source/book.o: source/book.cpp include/book.h
	g++ -std=c++17 -Iinclude -c source/book.cpp -o source/book.o

source/library.o: source/library.cpp include/library.h include/book.h
	g++ -std=c++17 -Iinclude -c source/library.cpp -o source/library.o

source/main.o: source/main.cpp include/library.h include/book.h
	g++ -std=c++17 -Iinclude -c source/main.cpp -o source/main.o

clean:
	rm -f source/book.o source/library.o source/main.o library-app

run:
	./library-app

.PHONY: all clean run

By running make then make run from Lab7/, does your program produce the correct output?

Error: missing separator

If you encounter the “missing separator” error when running make, it is possible that copying and pasting the makefile content into your editor replaced tabs with spaces. Make sure each command line in the makefile is preceded by a tab (Tab instead of four spaces).

Clean the project using make clean.

Automatic compilation¶

Go back to the previous makefile and try to understand it using the lecture. You can use the touch and make -d commands to better understand the different steps of the make process.

touch <file> and make -d

To better understand how make behaves, you can modify the modification date of existing files using the command touch <file>. If <file> does not exist, touch creates it; if it already exists, touch simply updates its last modification date.

Then, the command make -d displays all steps of the make process in the terminal. However, this command produces a lot of details that are not necessarily relevant. To only see the information we are interested in, you can use the following command:

make -d | grep -E 'Considering|Entering|Leaving|Finished|Prerequisite|No need|up to date'

grep -E '<regular expression>' allows you to retrieve only the lines that match the given regular expression. In this case, we are looking for all lines containing the specified keywords.

The command above only works if the terminal messages are in English. If they are in French, you can temporarily change the language in the terminal by running:

export LANG=en_US.UTF-8

std=c++17

We use c++17 to be able to use modern C++ syntax such as for (const Book& book : mBooks).

-Iinclude

-Iinclude is an option that tells the compiler that headers such as #include "book.h" are located in the include folder. This allows us to write #include "book.h" even though the header book.h is not located in the same place as the source code book.cpp.

In the prerequisites of the source files, we manually added the headers being used, but we want to be able to easily add or remove them without modifying our automatic compilation script (makefile). In other words, we want dependency generation to also be automated.

Run make clean before moving to the next step.
Apply the following changes to the makefile:

makefile

all: library-app

library-app: source/book.o source/library.o source/main.o
	g++ source/book.o source/library.o source/main.o -o library-app

source/book.o: source/book.cpp
	g++ -std=c++17 -Iinclude -MMD -MF source/book.d -c source/book.cpp -o source/book.o

source/library.o: source/library.cpp
	g++ -std=c++17 -Iinclude -MMD -MF source/library.d -c source/library.cpp -o source/library.o

source/main.o: source/main.cpp
	g++ -std=c++17 -Iinclude -MMD -MF source/main.d -c source/main.cpp -o source/main.o

clean:
	rm -f source/book.o source/library.o source/main.o source/book.d source/library.d source/main.d library-app

run:
	./library-app

.PHONY: all clean run

-include source/book.d source/library.d source/main.d

Run the make command and examine the contents of the generated .d files.

.d files (dependencies)

To avoid having to manually specify headers for the compilation of each source file, we automatically generate .d dependency files using the -MMD option (which includes prerequisites with the headers we created, but not those from stable C++ libraries), and we instruct make to include these prerequisites using the -include *.d option.

Due to execution order, the generation of main.d, for example, occurs after the prerequisites of main.o have been processed. This is not a problem, as the goal of adding headers to prerequisites is to avoid recompiling the entire project once it has already been compiled. The first compilation will generate these dependencies, and during subsequent compilations, the .d files will be included, allowing make to detect changes in headers.

-include *.d at the end

The -include directive must be placed at the end of the makefile (unlike regular include directives, which are placed at the beginning of the file). If -include is placed at the beginning, it overrides the default behavior of make, which is make all, by instead running make on the included targets.

The files generated during compilation are currently mixed with the source files. To keep our project clean, we will move them into another subdirectory called build/.

Run make clean before moving to the next step.
Apply the following changes to the makefile:

makefile

all: build/binaries/library-app

build:
	mkdir -p build/objects build/dependencies build/binaries

build/binaries/library-app: build/objects/book.o build/objects/library.o build/objects/main.o | build
	g++ build/objects/book.o build/objects/library.o build/objects/main.o -o build/binaries/library-app

build/objects/book.o: source/book.cpp | build
	g++ -std=c++17 -Iinclude -MMD -MF build/dependencies/book.d -c source/book.cpp -o build/objects/book.o

build/objects/library.o: source/library.cpp | build
	g++ -std=c++17 -Iinclude -MMD -MF build/dependencies/library.d -c source/library.cpp -o build/objects/library.o

build/objects/main.o: source/main.cpp | build
	g++ -std=c++17 -Iinclude -MMD -MF build/dependencies/main.d -c source/main.cpp -o build/objects/main.o

clean:
	rm -rf build

run:
	./build/binaries/library-app

.PHONY: all build clean run

-include build/dependencies/book.d build/dependencies/library.d build/dependencies/main.d

build/

To avoid mixing compilation files with the source code, we generate them in a directory called build and organize them into subdirectories: dependencies/ for .d files, objects/ for .o files, and binaries/ for executables.

To do this, we create a build target that generates the required directories using the -p option (for parents), which allows creating all directories (including parent directories) if they do not already exist. If the directories already exist, the command does not overwrite them.

The syntax | build indicates an order-only prerequisite. The processing of prerequisites is not necessarily ordered, which allows parallelization in large projects where multiple targets can be processed at the same time.
An order-only prerequisite is processed before the target, but it is not used to determine whether the target is up to date. In other words, | guarantees execution order, but does not influence recompilation.

Since we no longer generate all files in the same location, we specify with the -o option the location and name of the .o file to create. For example, -o build/objects/main.o indicates that main.o must be generated in build/objects/ with that name. The same applies to -MF build/dependencies/main.d for dependency files containing prerequisites.

Now, to clean the project (make clean), it is enough to delete the build/ directory. The -rf option of the rm command combines recursive and force, allowing directories and all their subdirectories to be deleted without confirmation.

We also add build to .PHONY in case the build/ directory exists but not its subdirectories. This could mislead make into thinking that build is up to date.

Run the different commands make, make run, and make clean, and observe the state of the directories after each command.
Run make clean before moving to the next step.

Ideally, we want a makefile that works for all our projects with the following structure:

project/
├── include/
│    ├── header.h
│    └── ...
├── source/
│    ├── code.cpp
│    ├── ...
│    └── ...
└── makefile

To achieve this, we will use variables to refactor the makefile.

Apply the following changes to the makefile:

makefile

EXECUTABLE = library-app

CXX = g++
CXXFLAGS = -std=c++17 -Iinclude

SOURCES = $(wildcard source/*.cpp)
OBJECTS = $(patsubst source/%.cpp,build/objects/%.o,$(SOURCES))
DEPENDENCIES = $(patsubst source/%.cpp,build/dependencies/%.d,$(SOURCES))

all: build/binaries/$(EXECUTABLE)

build:
	mkdir -p build/objects build/dependencies build/binaries

build/binaries/$(EXECUTABLE): $(OBJECTS) | build
	$(CXX) $(OBJECTS) -o build/binaries/$(EXECUTABLE)

build/objects/%.o: source/%.cpp | build
	$(CXX) $(CXXFLAGS) -MMD -MF build/dependencies/$*.d -c $< -o $@

clean:
	rm -rf build

run:
	./build/binaries/$(EXECUTABLE)

.PHONY: all build clean run

-include $(DEPENDENCIES)

Makefile variables

Our makefile variables will always be global for our use cases (therefore in UPPER_SNAKE_CASE). We will only use variables whose values are strings. VARIABLE is used to assign the variable, and $(VARIABLE) corresponds to its value.

EXECUTABLE: the name of the executable, the only element we will modify depending on the project; the rest of the makefile depends only on the project structure, not its content.
CXX: the C++ compiler we use (here g++, but others exist).
CXXFLAGS: compilation options (here -std=c++17 and -Iinclude). Often, we can add other options depending on our needs (for example, -Wall to enable warnings). Sometimes, we may also add linker options, in which case another variable must be defined.
SOURCES: the list of source code files (here $(SOURCES) is equal to the string source/book.cpp source/library.cpp source/main.cpp).
OBJECTS: the list of object files (here $(OBJECTS) is equal to the string build/objects/book.o build/objects/library.o build/objects/main.o).
DEPENDENCIES: the list of dependency files (here $(DEPENDENCIES) is equal to the string build/dependencies/book.d build/dependencies/library.d build/dependencies/main.d).

wildcard and patsubst

Here, the wildcard function lists all filenames in source/ that end with .cpp. The patsubst function takes two patterns (for example, source/%.cpp and build/objects/%.o) and replaces the first with the second (for example, source/main.cpp becomes build/objects/main.o) in the given string (for example, $(SOURCES)).

In our project, the following code

build/binaries/$(EXECUTABLE): $(OBJECTS) | build
	$(CXX) $(OBJECTS) -o build/binaries/$(EXECUTABLE)

becomes

build/binaries/library-app: build/objects/book.o build/objects/library.o build/objects/main.o | build
    g++ build/objects/book.o build/objects/library.o build/objects/main.o -o build/binaries/library-app

$<, $@, % and $*

The special variables $< and $@ represent the first prerequisite and the target, respectively. The variable % is used as before (in regular expressions) to represent a certain string in the pattern defining the target and can be reused in the prerequisites. The variable $* allows retrieving the value of % within commands.

In our project, the following code

build/objects/%.o: source/%.cpp | build
    $(CXX) $(FLAGS) -MMD -MF build/dependencies/$*.d -c $< -o $@

becomes

build/objects/main.o: source/main.cpp | build
    g++ -std=c++17 -Iinclude -MMD -MF build/dependencies/main.d -c source/main.cpp -o build/objects/main.o

for the target build/objects/main.o. The same rule applies to the other .o targets.

Go back to the objectives and check off the points you have mastered. Review the points you do not yet fully understand. Ask your instructor if needed.