PS4: Build system and documentation Hoang La
IUT d'Orsay, Université Paris-Saclay
Git integration in an IDE
Another session in the terminal!
We will create makefile files without an extension. Add !makefile to .gitignore to avoid ignoring them while continuing to exclude executable files.
Additionally, we will generate the directories build/ and docs/, which will contain compilation files and documentation that we will ignore.
The goal of this session is to understand the following points:
Exercise 1: Makefile ¶ Here is a minimal example of a C++ project:
minimal-project/
hello.h
hello.cpp
main.cpp
makefilehello.h
hello.cpp
main.cpp
makefile
#ifndef HELLO_H
#define HELLO_H
void sayHello();
#endif#include <iostream>
#include "hello.h"
void sayHello() {
std::cout << "Hello World!" << std::endl;
}#include "hello.h"
int main() {
sayHello();
return 0;
}all: executable
executable: main.o hello.o
g++ -o executable main.o hello.o
main.o: main.cpp
g++ -c main.cpp
hello.o: hello.cpp
g++ -c hello.cpp
run:
./executable
clean:
rm -f executable main.o hello.o
.PHONY: all run cleanInstead of manually compiling and executing each project, as projects become more complex, they require a build system based on build scripts .
A build script is a file that automates the compilation process and dependency management of a project. It defines the necessary steps to build the project, such as compiling source code, assembling files, generating documentation, or running tests. Its purpose is to simplify, standardize, and accelerate the project build process.
The standard build systems in C++ are CMake and Makefile . Other languages use their own tools, such as MSBuild for C# or Gradle and Maven for Java. Some IDEs also integrate their own build systems. For example, a Java project in Eclipse uses Eclipse’s native build system.
In this session, we will explore the operation of a build system through a Makefile build script, a low-level build system for C/C++ designed for Unix systems (Linux and macOS). CMake is more high-level and cross-platform (compatible with other systems like Windows), but we will focus on Makefile , which provides better low-level control.
Linux on Windows
If you are using Windows at the IUT, you can try using Git Bash and hope that the necessary installations are available. If that does not work, use Linux (Debian distribution at the IUT)...
If you are using your own machine, you can install Windows Subsystem for Linux (WSL)
Create the directory and files listed above in PS4/ with the corresponding code. A Makefile uses a syntax based on targets . In the terminal, you can execute a target with the command:
Here are some common examples:
make executes the default allmake run runs the compiled program (the executable).make clean removes the files generated by the compilation to clean the project.By default, make looks for a file named makefile or Makefile to find the targets to execute. If your build script has a different name, like my-build-script, you will need to specify its name with the -f option:
make -f my-build-script <target>-f stands for file , indicating a specific makefile.
Behavior of make <target>
make <target> attempts to create a file with the name of the target by executing the associated commands.
Before building a target, make attempts to build its prerequisites . If everything is up to date, meaning that for every (target, prerequisite) pair involved in the target’s construction, the target file has been created/modified after the prerequisite file, then make does nothing. If the file does not exist, make considers it outdated.
When a target is outdated, make executes the commands associated with it.
For example:
target : prerequisite
target command
prerequisite :
prerequisite commandIn the example above, if both target and prerequisite files exist, and the last modification of target was at 2:00 PM while prerequisite was modified at 1:59 PM, then make checks that prerequisite is up to date (which it is since it has no prerequisites itself). Then it verifies if target is up to date (which it is, since its last modification is more recent than prerequisite) and does nothing.
This behavior optimizes compilation, avoiding recompiling the entire project, which is crucial for large projects that may take hours to compile.
Explanation of the makefile from the example :
all: This is the default target executed when typing make. It contains no command and simply checks if executable is up to date.executable: This target depends on main.o and hello.o, which are object files generated from the project’s .cpp files. Once these prerequisites are up to date, make executes g++ -o executable main.o hello.o, which links main.o and hello.o to create the executable binary (Linking, the final step of C++ compilation).main.o and hello.o:main.o depends on main.cpp. Even though main.cpp is not an explicit target, make simply checks if the file exists and is up to date.To generate main.o, make executes g++ -c main.cpp. This runs the preprocessor , generates the translation unit , and compiles the code into an object file .o. The same logic applies to hello.o. run: Assumes the program is already compiled and simply executes ./executable.clean:Has no prerequisites. Deletes generated files using rm -f executable main.o hello.o. The -f (force ) option removes files without confirmation and without errors if the files do not exist. This option is commonly used for clean in makefiles. You may have noticed that some targets (all, run, clean) do not correspond to files, and the associated commands do not generate files or directories with these names.
Since these files do not exist, make always considers these targets as outdated and executes them systematically, which is the intended behavior for all, run, and clean.
Potential issue:
If the project contains files named all, run, or clean, make might interpret them as up-to-date files and not execute the associated commands.
Solution: .PHONY targets
To prevent this confusion, we declare these targets as phony using the directive .PHONY: all run clean.
This tells make that all, run, and clean are not files but commands to execute systematically, even if files with these names exist.
Why separate compilation into multiple steps?
We are used to compiling in a single step without generating intermediate files like .o files. Why not do the same here?
In a growing project, different parts of the code that make up the final executable(s) arrive gradually. It is essential to be able to compile and test certain parts of the code without waiting for the entire project to be completed.
Even if all files are present, modifying a single part of the code does not mean we need to recompile everything. For large projects, where compilation can take minutes (or even hours), avoiding full recompilation saves a significant amount of time.
Thus, splitting compilation into multiple steps (generating .o files for each .cpp file, then final linking) is a good practice for evolving and large projects.
Run make, make run, and make clean, and observe their effects in the terminal and on the files. Error: missing separator
If you encounter the “missing separator” error when running make, it is possible that copying and pasting the makefile content into your editor replaced tab characters with spaces. Make sure that each command line in the makefile is preceded by a tab (Tab instead of four spaces).
In the previous makefile, there are implicit prerequisites, which are the .h header files. For example, main.cpp and hello.cpp depend on hello.h, but this prerequisite is not specified in the makefile. This does not cause issues during compilation because g++, thanks to the #include directives, understands that it must include the appropriate headers during the translation of the .cpp source code.
A problem can arise when a header is modified, and we want to recompile only part of the project instead of recompiling everything. Since headers are not explicitly listed as prerequisites, if all other files exist and their modification date does not indicate any changes, make would not recompile the project. Therefore, it is necessary to add these prerequisites to the makefile, even if they do not appear in the compilation commands.
To better understand the behavior of make, you can modify the modification date of existing files using the touch <file> command. If <file> does not exist, touch creates it; if it already exists, touch simply updates its last modification date.
Then, the make -d command displays all the steps of the make process in the terminal. However, this command generates a lot of details that are not necessarily relevant. To see only the information that interests us, you can use the following command:
make -d | grep -E 'Considering|Entering|Leaving|Finished|Prerequisite|No need|up to date'grep -E '<regular expression>' allows you to retrieve only the lines that match the given regular expression. In this case, we are looking for all lines containing the specified keywords.
The above command only works if your terminal messages are in English.
If they are in French, you can change the terminal language temporarily by running:
Make the following modifications to the makefile: all: executable
executable: main.o hello.o
g++ -o executable main.o hello.o
main.o: main.cpp hello.h
g++ -c main.cpp
hello.o: hello.cpp hello.h
g++ -c hello.cpp
run:
./executable
clean:
rm -f executable main.o hello.o
.PHONY: all run cleanRun make, make run, and make clean again. The behavior of these commands should not change.
Make the following modifications to the makefile:
all: executable
executable: main.o hello.o
g++ -o executable main.o hello.o
main.o: main.cpp
g++ -c main.cpp -MMD
hello.o: hello.cpp
g++ -c hello.cpp -MMD
run:
./executable
clean:
rm -f executable main.o hello.o main.d hello.d
.PHONY: all run clean
-include *.dRun make and examine the content of the generated .d files. To avoid manually specifying headers for each source file compilation, we automatically generate .d dependency files using the -MMD option (which includes prerequisites with the headers we created, but not those from stable C++ libraries) and instruct make to include these prerequisites with the -include *.d option.
Due to the order of execution, generating main.d, for example, occurs after the prerequisites for main.o have been processed. This is not a problem because the purpose of adding headers as prerequisites is to prevent recompiling the entire project once it has already been compiled. The first compilation will generate these dependencies, and in subsequent compilations, the .d files will be included, allowing make to detect changes in the headers.
Make the following modifications to the makefile: all: build build/binaries/executable
build:
mkdir -p build/dependencies build/objects build/binaries
build/binaries/executable: build/objects/main.o build/objects/hello.o
g++ -o build/binaries/executable build/objects/main.o build/objects/hello.o
build/objects/main.o: main.cpp
g++ -c main.cpp -o build/objects/main.o -MMD -MF build/dependencies/main.d
build/objects/hello.o: hello.cpp
g++ -c hello.cpp -o build/objects/hello.o -MMD -MF build/dependencies/hello.d
run:
./build/binaries/executable
clean:
rm -rf build
.PHONY: all run clean
-include build/dependencies/*.dTo avoid mixing compilation files with source code, we generate them in a directory named build and organize them into subdirectories: dependencies/ for .d files, objects/ for .o files, and binaries/ for executables.
To do this, we create a build target that generates the necessary directories using the -p option (for parents), allowing all required directories (including parent directories) to be created if they do not already exist. If the directories exist, the command will not overwrite them.
In the prerequisites of the all target, we add build first to ensure it runs before make looks for prerequisite files in build.
Since we are no longer generating all files in the same location, we specify the destination and filename for each .o file using the -o option. For example, -o build/objects/main.o ensures main.o is generated in build/objects/. Similarly, -MF build/dependencies/main.d specifies the dependency file location and its name.
Now, to clean the project (make clean), we simply delete the build/ directory. The -rf option in rm combines recursive and force , allowing directory and subdirectory deletion without confirmation.
We also add build to .PHONY in case the build/ directory exists, but not its subdirectories. This could mislead make, making it think that build is up to date.
Run make, make run, and make clean again and observe.
Add comments to the makefile (using #) to clarify the discussed points.
Large project organization
Large projects are organized with multiple makefiles (one at the project root and one per module).
minimal-project/
source/
first-module/
hello.cpp
hello.h
makefile
second-module/
hi.cpp
hi.h
makefile
main.cpp
makefileWe will explore compilation management in this case in the following bonus section.
Bonus ¶ The current makefile code contains a lot of redundancies. We can refactor this code using variables as follows:
BUILD_DIRECTORY = build
DEPENDENCY_DIRECTORY = $(BUILD_DIRECTORY)/dependencies
OBJECT_DIRECTORY = $(BUILD_DIRECTORY)/objects
BINARY_DIRECTORY = $(BUILD_DIRECTORY)/binaries
EXECUTABLE = $(BINARY_DIRECTORY)/executable
SOURCE_FILES = $(wildcard *.cpp)
OBJECT_FILES = $(SOURCE_FILES:.cpp=.o)
OBJECT_FILES := $(OBJECT_FILES:%=$(OBJECT_DIRECTORY)/%)
DEPENDENCY_FILES = $(SOURCE_FILES:.cpp=.d)
DEPENDENCY_FILES := $(DEPENDENCY_FILES:%=$(DEPENDENCY_DIRECTORY)/%)
all: $(BUILD_DIRECTORY) $(EXECUTABLE)
$(BUILD_DIRECTORY):
mkdir -p $(DEPENDENCY_DIRECTORY) $(OBJECT_DIRECTORY) $(BINARY_DIRECTORY)
$(EXECUTABLE): $(OBJECT_FILES)
g++ -o $@ $^
$(OBJECT_DIRECTORY)/%.o: %.cpp
g++ -c $< -o $@ -MMD -MF $(DEPENDENCY_DIRECTORY)/$*.d
run:
./$(EXECUTABLE)
clean:
rm -rf $(BUILD_DIRECTORY)
.PHONY: all run clean $(BUILD_DIRECTORY)
-include $(DEPENDENCY_FILES)Variables in a makefile are named following the UPPER_SNAKE_CASE convention. Their values are accessed using $(VARIABLE).
Here, we have defined variables for different directory names and file types. Let’s look at the content of this code:
BUILD_DIRECTORY refers to the directory that will contain our compilation files: here, build.Within build, we organize subdirectories for different file types mentioned earlier. For example, the value of OBJECT_DIRECTORY is build/objects. EXECUTABLE allows us to define the name of the executable located in build/binaries/. It is important to differentiate between the file name and the file path from the current directory. $(EXECUTABLE) will be used to generate the file in the correct location using a path, and it is not just the executable name.SOURCE_FILES is the list of paths to source files. Here, since these files are located in the same directory as the makefile, we can obtain these paths using the wildcard *.cpp command, which returns main.cpp hello.cpp. We will reorganize these source files into subdirectories later.OBJECT_FILES is the list of paths to object files. We obtain this list first with OBJECT_FILES = $(SOURCE_FILES:.cpp=.o), which iterates over main.cpp hello.cpp and replaces the .cpp extension with .o to get main.o hello.o. The instruction OBJECT_FILES := $(OBJECT_FILES:%=$(OBJECT_DIRECTORY)/%) iterates over this list main.o hello.o and replaces the % pattern (e.g., main.o) with $(OBJECT_DIRECTORY)/%, resulting in build/objects/main.o. Thus, OBJECT_FILES becomes build/objects/main.o build/objects/hello.o. Using := instead of = ensures that make evaluates the expression only once, preventing infinite loops caused by OBJECT_FILES appearing on both sides of the definition.The same applies to DEPENDENCY_FILES. The rest of the code is similar to the previous one, with variables replacing hardcoded values:
$@ returns the value of the current target.$^ returns the value of the prerequisites.$< returns the value of the first prerequisite.$* returns the value of the current % pattern.For example:
$(EXECUTABLE): $(OBJECT_FILES)
g++ -o $@ $^Here, $@ returns $(EXECUTABLE), which is build/binaries/executable, and $^ returns $(OBJECT_FILES), which is build/objects/main.o build/objects/hello.o. Thus, the command g++ -o $@ $^ becomes g++ -o build/binaries/executable build/objects/main.o build/objects/hello.o.
Another example:
$(OBJECT_DIRECTORY)/%.o: %.cpp
g++ -c $< -o $@ -MMD -MF $(DEPENDENCY_DIRECTORY)/$*.dThe pattern $(OBJECT_DIRECTORY)/%.o defines targets in the form build/objects/<filename>.o and repeats the % pattern in the prerequisites with %.cpp. Thus, build/objects/main.o has main.cpp as its prerequisite since % here corresponds to main.build/objects/main.o, $< is main.cpp, $@ is build/objects/main.o, and $(DEPENDENCY_DIRECTORY)/$*.d is build/dependencies/main.d, since $* takes the value of %, which is main.g++ -c $< -o $@ -MMD -MF $(DEPENDENCY_DIRECTORY)/$*.d becomes g++ -c main.cpp -o build/objects/main.o -MMD -MF build/dependencies/main.d, which is the same line of code as before.build/objects/hello.o.
Run make, make run, and make clean again and observe. We can still go further in refactoring by creating a variable for the compiler g++, in case we want to change compilers in the future, or a variable for the compilation options -c -o -MMD -MF <...>, which could even vary depending on the context.
Unfortunately, that is not the case yet!.o targets, which are all in build/objects/, but depend on .cpp files from different directories.
Depending on the module, it may also be desirable to compile the project differently or only compile a single module. For example, if other teams are working on the other modules, or if the project is too large to run a full make just to test one module.
The best practice here is to have multiple makefiles, one for each module, and a main makefile that will call the makefiles of the different modules.
Rearrange your minimal project as follows: minimal-project/
source/
first-module/
hello.cpp
hello.h
makefile
second-module/
hi.cpp
hi.h
makefile
main.cpp
makefileIn hi.h, copy the code from hello.h and modify the include guard and the function name to sayHi.
In hi.cpp, copy the code from hello.cpp and modify the include directive to include hi.h, the function name to sayHi, and the console output to Hi! instead of Hello World!.
In main.cpp, modify the include directives to include first-module/hello.h and second-module/hi.h and add sayHi(); after sayHello();.
For this minimal example, we will use the same makefile in both modules, but we can imagine that these makefiles might differ and even be written by different people.
Modify the module makefiles as follows: BUILD_DIRECTORY = ../../build
DEPENDENCY_DIRECTORY = $(BUILD_DIRECTORY)/dependencies
OBJECT_DIRECTORY = $(BUILD_DIRECTORY)/objects
SOURCE_FILES = $(wildcard *.cpp)
OBJECT_FILES = $(SOURCE_FILES:.cpp=.o)
OBJECT_FILES := $(OBJECT_FILES:%=$(OBJECT_DIRECTORY)/%)
DEPENDENCY_FILES = $(SOURCE_FILES:.cpp=.d)
DEPENDENCY_FILES := $(DEPENDENCY_FILES:%=$(DEPENDENCY_DIRECTORY)/%)
all: $(BUILD_DIRECTORY) $(OBJECT_FILES)
$(BUILD_DIRECTORY):
mkdir -p $(DEPENDENCY_DIRECTORY) $(OBJECT_DIRECTORY) $(BINARY_DIRECTORY)
$(OBJECT_DIRECTORY)/%.o: %.cpp
g++ -c $< -o $@ -MMD -MF $(DEPENDENCY_DIRECTORY)/$*.d
clean:
rm -rf $(OBJECT_FILES) $(DEPENDENCY_FILES)
.PHONY: all clean $(BUILD_DIRECTORY)
-include $(DEPENDENCY_FILES)The code remains unchanged without generating an executable. The build directory will be created in the grandparent directory (../../build), at the same level as source. The default target all will create the $(OBJECT_FILES) instead of the executable. The clean target deletes only the object and dependency files related to the module. Modify the root project makefile as follows: BUILD_DIRECTORY = build
DEPENDENCY_DIRECTORY = $(BUILD_DIRECTORY)/dependencies
OBJECT_DIRECTORY = $(BUILD_DIRECTORY)/objects
BINARY_DIRECTORY = $(BUILD_DIRECTORY)/binaries
SOURCE_DIRECTORY = source
EXECUTABLE = $(BINARY_DIRECTORY)/executable
MODULES = $(wildcard $(SOURCE_DIRECTORY)/*/)
SOURCE_FILES = $(wildcard $(SOURCE_DIRECTORY)/*.cpp)
OBJECT_FILES = $(SOURCE_FILES:$(SOURCE_DIRECTORY)/%.cpp=%.o)
OBJECT_FILES := $(OBJECT_FILES:%=$(OBJECT_DIRECTORY)/%)
DEPENDENCY_FILES = $(SOURCE_FILES:$(SOURCE_DIRECTORY)/%.cpp=%.d)
DEPENDENCY_FILES := $(DEPENDENCY_FILES:%=$(DEPENDENCY_DIRECTORY)/%)
all: $(BUILD_DIRECTORY) $(EXECUTABLE)
$(BUILD_DIRECTORY):
mkdir -p $(DEPENDENCY_DIRECTORY) $(OBJECT_DIRECTORY) $(BINARY_DIRECTORY)
$(EXECUTABLE): $(MODULES) $(OBJECT_FILES)
g++ -o $@ $(wildcard $(OBJECT_DIRECTORY)/*.o)
$(MODULES):
$(MAKE) -C $@
$(OBJECT_DIRECTORY)/%.o: $(SOURCE_DIRECTORY)/%.cpp
g++ -c $< -o $@ -MMD -MF $(DEPENDENCY_DIRECTORY)/$*.d
run:
./$(EXECUTABLE)
clean:
rm -rf $(BUILD_DIRECTORY)
.PHONY: all run clean $(MODULES) $(BUILD_DIRECTORY)
-include $(DEPENDENCY_FILES)We retrieve the source code directory (source/). Inside source/, we retrieve the module names that correspond to subdirectories. We also retrieve the .cpp source code files. In this example, we only have main.cpp, but there could be other files. Similar to before, we retrieve the names of object files and the dependency files to generate. To create the final executable, we first compile the modules and the object files from the new source code, then link all the object files in build/objects/. For each module (source/first-module source/second-module), we go into the corresponding subdirectory (e.g., source/first-module) with the -C source/first-module option. Then, we run make via the native $(MAKE) variable, which inherits the options from the make command in our terminal. For example, we can parallelize the compilation when our machine has multiple processors, as source files can be compiled simultaneously. So, make -j4 uses 4 processors at once, and $(MAKE) will take the -j4 option to compile the modules. For our example, we could replace $(MAKE) with make directly, but it is better in general to use the native variable associated with the make command. Add comments to the makefiles with # to clarify the different blocks of code and add examples for the commands to clarify the syntax. For example:
# Compile source codes to object files
$(TARGET_VARIABLE): $(PREREQUISITE_VARIABLE)
# For example: g++ -c prerequisite.cpp -o target.o
g++ -c $< -o $@Exercise 2: Documentation ¶ Rewrite your README and check the rendering of the README on GitLab. The main features of this project are to learn good coding and development practices. There is no additional documentation. You can explain how to clone the repository by creating a PAT (it’s not necessary to be as detailed as in the first TP) and how to compile the code via the command line with g++ for installation instructions. You can explain how to run an executable from the command line for usage instructions. You can add an MIT license in a separate file (named LICENSE) and include a link to it at the root of the project. Some Markdown syntax
# creates headings. The more #, the smaller the heading.[text](link) creates hyperlinks.``` code ``` creates code blocks.**bold text** makes the text bold.*italic text* makes the text italic.- creates bullet points.1., 2., etc. creates numbered lists.> quote followed by > - author creates a quote with an author. inserts an image and shows the alternative text when the image fails to load or when we hover over it.- [ ] creates a checkbox, and - [x] a checked box.--- inserts a horizontal line.Create a documentation/ directory in PS4/. In class, we saw an example of documentation using Doxygen (very similar to Javadoc syntax). Since Doxygen is not (yet) installed on the IUT machines, we will review the same example in Javadoc.
Create the following file. /**
* Represents a temperature in Celsius and Fahrenheit.
*
* This class allows you to set a temperature in Celsius, convert it to Fahrenheit,
* and vice versa. It also provides the ability to get the current temperature in either unit.
*
* <p> Example usage: </p>
* <pre>
* <code>
* Temperature currentTemperature = new Temperature(25.0); // 25°C
* currentTemperature.displayTemperature(); // Output: Temperature: 25°C / 77°F
* currentTemperature.setFahrenheit(100.0);
* currentTemperature.displayTemperature(); // Output: Temperature: 37.7778°C / 100°F
* </code>
* </pre>
*/
public class Temperature {
private double celsius;
/**
* Constructs a new Temperature object.
*
* Initializes the temperature in Celsius.
*
* @param celsius The initial temperature in Celsius.
*/
public Temperature(double celsius) {
this.celsius = celsius;
}
/**
* Gets the current temperature in Celsius.
*
* @return The temperature in Celsius.
*/
public double getCelsius() {
return celsius;
}
/**
* Sets the temperature in Celsius.
*
* This method sets the temperature value directly in Celsius.
*
* @param celsius The new temperature in Celsius.
*/
public void setCelsius(double celsius) {
this.celsius = celsius;
}
/**
* Gets the current temperature in Fahrenheit.
*
* @return The temperature in Fahrenheit.
*/
public double getFahrenheit() {
return celsius * 9 / 5 + 32; // Formula to convert Farenheit to Celsius
}
/**
* Sets the temperature using a value in Fahrenheit.
*
* This method converts the given Fahrenheit value to Celsius and sets it.
*
* @param fahrenheit The temperature in Fahrenheit.
*/
public void setFahrenheit(double fahrenheit) {
this.celsius = (fahrenheit - 32) * 5 / 9; // Formula to convert Celsius to Farenheit
}
/**
* Converts and displays the temperature in both Celsius and Fahrenheit.
*
* This method prints the current temperature in both Celsius and Fahrenheit.
*/
public void displayTemperature() {
System.out.println("Temperature: " + celsius + "°C / " + getFahrenheit() + "°F");
}
}Generate the documentation for Temperature.java with the command javadoc -d docs Temperature.java, which will create a docs/ directory.
View the generated documentation by opening docs/index.html in a web browser.
Revisit the Doxygen syntax covered in class to write the documentation for the Product class from the short-functions exercise in TP3.
@example in Doxygen
The @example tag in Doxygen should be used in a separate block. Otherwise, Doxygen will treat the entire block as an example and will not show the descriptions in the generated documentation, unlike Javadoc which does not have an @example tag.
Return to the objectives