Modern C++
Programming
1. Introduction
Federico Busato
2024-03-29
About Motivation 1/5
“When recruiting research assistants, I look at grades as the last indi-
cator. I find that imagination, ambition, initiative, curiosity, drive,
are far better predictors of someone who will do useful work with me. Of
course, these characteristics are themselves correlated with high grades,
but there is something to be said about a student who decides that a
given course is a waste of time and that he works on a side project in-
stead.
Breakthroughs don’t happen in regular scheduled classes, they happen
in side projects. We want people who complete the work they were as-
signed, but we also need people who can reflect critically on what
is genuinely important”
Daniel Lemire, Prof. at the University of Quebec
2/56
About Motivation 2/5
Academic excellence is not a strong predictor
of career excellence
“Across industries, research shows that the correlation between grades
and job performance is modest in the first year after college and trivial
within a handful of years...
Academic grades rarely assess qualities like creativity, leadership and team-
work skills, or social, emotional and political intelligence. Yes, straight-A
students master cramming information and regurgitating it on exams.
But career success is rarely about finding the right solution to a
problem — it’s more about finding the right problem to solve...”
3/56
About Motivation 3/5
“Getting straight A’s requires conformity. Having an influential
career demands originality.
This might explain why Steve Jobs finished high school with a 2.65
G.P.A., J.K. Rowling graduated from the University of Exeter with
roughly a C average, and the Rev. Dr. Martin Luther King Jr. got only
one A in his four years at Morehouse
If your goal is to graduate without a blemish on your transcript, you
end up taking easier classes and staying within your comfort zone. If
you’re willing to tolerate the occasional B...You gain experience coping
with failures and setbacks, which builds resilience”
4/56
About Motivation 4/5
“Straight-A students also miss out socially. More time studying in
the library means less time to start lifelong friendships, join new clubs or
volunteer...Looking back, I don’t wish my grades had been higher. If I
could do it over again, I’d study less”
Adam Grant, the New York Times
www.nytimes.com/2018/12/08/opinion/college-gpa-career-success.html
5/56
About Motivation 5/5
“Got a 2.4 GPA my first semester in college. Thought maybe I wasn’t
cut out for engineering. Today I’ve landing two spacecraft on Mars, and
designing one for the moon.
STEM is hard for everyone. Grades ultimately aren’t what matters.
Curiosity and persistence matter”
Ben Cichy, Chief Software Engineer,
NASA Mars Science Laboratory
twitter.com/bencichy/status/1197752802929364992?s=20
6/56
About Programming 1/2
“And programming computers was so fascinating. You create your
own little universe, and then it does what you tell it to do”
Vint Cerf, TCP/IP co-inventor and Turing Award
“Most good programmers do programming not because they expect to
get paid or get adulation by the public, but because it is fun to program”
Linus Torvalds, principal developer of the Linux kernel
“You might not think that programmers are artists, but programming
is an extremely creative profession. It’s logic-based creativity”
John Romero, co-founder of id Software
7/56
About Programming 2/2
Creativity Programming is extremely creative. The ability to perceive the problem in
a novel way, provide new and original solutions. Creativity allows
recognizing and generating alternatives
Form of Art Art is the expression of human creative skills. Every programmer has his
own style. Codes and algorithms show elegance and beauty in the same
way as painting or music
Learn Programming gives the opportunity to learn new things every day,
improve own skills and knowledge
Challenge Programming is a challenge. A challenge against yourself, the problem,
and the environment
8/56
Knowledge-Experience Relation
9/56
Learning
“In software development, learning is not a big part of the job.
It is the job.”
Woody Zuill
10/56
The Assembly Programming Language
A long time ago, in a galaxy far,
far away....there was Assembly
• Extremely simple instructions
• Requires lots of code to do simple tasks
• Can express anything your computer can do
• Hard to read, write
• ...redundant, boring programming, bugs pro-
liferation
main:
.Lfunc_begin0:
push rbp
.Lcfi0:
.Lcfi1:
mov rbp, rsp
.Lcfi2:
sub rsp, 16
movabs rdi, .L.str
.Ltmp0:
mov al, 0
call printf
xor ecx, ecx
mov dword ptr [rbp - 4], eax
mov eax, ecx
add rsp, 16
pop rbp
ret
.Ltmp1:
.Lfunc_end0:
.L.str:
.asciz "Hello World\n"
11/56
A Little History of C 1/3
In the 1969 Dennis M. Ritchie and Ken Thompson (AT&T, Bell Labs) worked on
developing an operating system for a large computer that could be used by a thousand
users. The new operating system was called UNIX
The whole system was still written in assembly code. Besides assembler and Fortran,
UNIX also had an interpreter for the programming language B. A high-level language
like B made it possible to write many pages of code task in just a few lines of code. In
this way the code could be produced much faster than in assembly
A drawback of the B language was that it did not know data-types (everything was
expressed in machine words). Another functionality that the B language did not provide
was the use of “structures”. The lack of these things formed the reason for Dennis
M. Ritchie to develop the programming language C. In 1988 they delivered the final
standard definition ANSI C
12/56
A Little History of C 2/3
Dennis M. Ritchie and Ken Thompson
#include "stdio.h"
int main() {
printf("Hello World\n");
}
13/56
A Little History of C 3/3
Areas of Application:
• UNIX operating system
• Computer games
• Due to their power and ease of use, C were used in the programming of the
special effects for Star Wars
Star Wars - The Empire Strikes Back
14/56
A Little History of C++ 1/3
The C++ programming language (originally named “C with Classes”) was devised
by Bjarne Stroustrup also an employee from Bell Labs (AT&T). Stroustrup started
working on C with Classes in 1979. (The ++ is C language operator)
The first commercial release of the C++ language was in October 1985
15/56
A Little History of C++ 2/3
The roots of C++
“The Evolution of C++Past, Present, and Future”, B. Stroustrup, CppCon16
16/56
A Little History of C++ 3/3
17/56
About Evolution
“If you’re teaching today what you were teaching five
years ago, either the field is dead or you are”
Noam Chomsky
18/56
Why C++ is so Popular? 1/2
There may be more than 200 billion lines
of C/C++ code globally
• Performance is the defining aspect of C++. No other programming
language provides the performance-critical facilities of C++
• Provide the programmer control over every aspect of performance
• Leave no room for a lower level language
Total number of lines of all code in use?
22/56
Why C++ is so Popular? 2/2
• Ubiquity. C++ can run from a low-power embedded device to large-scale
supercomputers
• Multi-Paradigm. Allow writing efficient code without losing high-level
abstraction
• Allow writing low-level code. Drivers, kernels, assembly (asm), etc.
• Ecosystem. Many support tools such as debuggers, memory checkers,
coverage, static analysis, profiling, etc.
• Maturity. C++ has a 40 years history. Many software problems have been
already addressed and developing practices have been investigated
23/56
Areas of Application 1/2
• Operating systems: Windows, Android, OS X, Linux
• Compilers: LLVM, Swift compiler
• Artificial Intelligence: TensorFlow, Caffe, Microsoft Cognitive Toolkit
• Image Editing: Adobe Premier, Photoshop, Illustrator
• Web browser: Firefox, Chrome, etc. + WebAssembly
• High-Performance Computing: drug developing and testing, large scale climate
models, physic simulations
• Embedded systems: IoT, network devices (e.g. GSM), automotive
• Google and Microsoft use C++ for web indexing
24/56
Areas of Application 2/2
• Scientific Computing: CERN/NASA*, SETI@home, Folding@home
• Database: MySQL, ScyllaDB
• Video Games: Unreal Engine, Unity
• Entertainment: Movie rendering (see Interstellar black hole rendering),
virtual reality
• Finance: electronic trading systems (Goldman, JPMorgan, Deutsche Bank)**
... and many more
* The flight code of the NASA Mars drone for the Perseverance Mission, as well as the Webb
telescope software, are mostly written in C++ github.com/nasa/fprime, James Webb Space
Telescope’s Full Deployment
** C++ is the new Python
25/56
Why C++ is so Important?
The End of Historical Performance Scaling
Performance limitations influence algorithm design
and research directions
26/56
An Important Example... (AI Evolution)
27/56
Performance 1/4
8.23
21.47
21.96 22.1
26.61
300
360
660
780
0
100
200
300
400
500
600
700
800
900
C++ GO SWIFT JAVA Node.js PHP Ruby Perl Python3
Execution TIme (S)
Programming Language
N-BODY SIMULATION
P R O G R A M M I N G L A N G UA G E S P E R F O R M A N C E C O M PA R I S O N
28/56
Performance 3/4
”A New Golden Age for Computer Architecture“, J. L. Heneessey, D. A. Patterson, 2019
30/56
Performance/Expressiveness Trade-off
1
10
100
1,000
10,000
100,000
1,000,000
Assembly C C++ Java JS Python
INSTRUCTIONS PER LINE
Mandelbrot Static Instructions per Line
32/56
C++ Philosophy - Performance 1/3
Do not sacrifice performance except as a last resort
Zero Overhead Principle (zero-cost abstraction)
“it basically says if you have an abstraction it should not
cost anything compared to write the equivalent code at lower
level”
“so I have say a matrix multiply it should be written in a
such a way that you could not drop to the C level of abstrac-
tion and use arrays and pointers and such and run faster”
Bjarne Stroustrup
36/56
C++ Philosophy - Type Safety 2/3
Enforce safety at compile time whenever possible
Statically Typed Language
“The C++ compiler provides type safety and catches
many bugs at compile time instead of run time (a critical
consideration for many commercial applications.)”
www.python.org/doc/FAQ.html
• The type annotation makes the code more readable
• Promote compiler optimizations and runtime efficiency
• Allow users to define their own type system
37/56
C++ Philosophy 3/3
• Programming model: compartmentalization, only add
features if they solve an actual problem, and allow full control
• Predictable runtime (under constraints): no garbage
collector, no dynamic type system → real-time systems
• Low resources: low memory and energy consumption →
restricted hardware platforms
• Well suited for static analysis → safety critical software
• Portability → Modern C++ standards are highly portable
38/56
Who is C++ for?
“C++ is for people who want to use hardware very well
and manage the complexity of doing that through abstrac-
tion”
Bjarne Stroustrup
“a language like C++ is not for everybody. It is gener-
ated via sharp and effective tool for professional basically and
definitely for people who aim at some kind of precision”
Bjarne Stroustrup
39/56
Why C++ is so Difficult? 1/2
... and why teaching C++ as first programming language is a bad idea?
C++ is the hardest language from students to master
• More languages in one
- Standard C/C++ programming
- Preprocessor
- Object-Oriented features
- Templates and Meta-Programming
• Huge set of features
• Worry about memory management
• Low-level implementation details: pointer arithmetic, structure, padding,
undefined behavior, etc.
• Frustrating: compiler/runtime errors (e.g. seg. fault)
41/56
Why C++ is so Difficult? 2/2
“C makes it easy to shoot yourself in the foot; C++ makes it harder,
but when you do, it blows your whole leg off”
Bjarne Stroustrup, Creator of the C++ language
“The problem with using C++...is that there’s already a strong ten-
dency in the language to require you to know everything before you can
do anything”
Larry Wall, Creator of the Perl language
“Despite having 20 years of experience with C++, when I compile a
non-trivial chunk of code for the first time without any error or warning,
I am suspicious. It is not, usually, a good sign”
Daniel Lemire, Prof. at the University of Quebec
42/56
C++ Weaknesses 1/2
Backward-compatibility
‘‘Dangerous defaults and constructs, often originating from C, cannot be removed
or altered”
“Despite the hard work of the committee, newer features sometimes have flaws
that only became obvious after extensive user experience, which cannot then be
fixed”
“C++ practice has put an ever-increasing cognitive burden on the developer for
what I feel has been very little gain in productivity or expressiveness and at a huge cost
to code clarity”
43/56
C++ Alternatives: Rust
Rust (1.0, 2015) has been Stack Overflow’s most loved language for eight years in a
row. Rust focuses on performance and zero-abstraction overhead as C++. It is
designed to prevent many vulnerabilities that affect C++, especially memory bugs,
enforcing constraints at compile type. In addition, it promotes cross-platform
compatibility
“First-time contributors to Rust projects are about 70 times less likely to
introduce vulnerabilities than first-time contributors to C++ projects”
Tracey et al.
1
1
Grading on a Curve: How Rust can Facilitate New Contributors while Decreasing
Vulnerabilities
• CISA, NSA: The Case for Memory Safe Roadmap
• Octoverse: The Fastest Growing Languages
• Secure by Design: Google’s Perspective on Memory Safety
45/56
C++ Alternatives: Zig
Zig (2016) is a minimal open-source programming language that can be intended as
replacement of C. Zig supports compile time generics, reflection and evaluation,
cross-compiling, and manual memory management. It is made to be fully interoperable
with C and also includes a C/C++ compiler.
Zig Programming Language
46/56
Why Switching to a New Language is Hard? 1/3
• No perfect language. There are always newer ’shining’ languages
• Alignment. Force all developers to switch to the new language
• Interoperability. Hundreds of billion lines of existing code. Must interoperate
with C and C++ code imposing serious design constraints
• Ecosystem. Lack of tools and libraries developed in the last four decades
• Time and Cost. Converting a codebase of 10 million lines: 500 developers, 5
years, $1,400,000,000
1
1
Bjarne Stroustrup: Delivering Safe C++
47/56
Why Switching to a New Language is Hard? 3/3
49/56
Language Complexity
Every second spent trying to understand the
language is one not spent understanding the
problem
50/56
The Course
51/56
The Course
Don’t forget: The right name of the course should be
“Introduction to Modern C++ Programming”
For many topics in the course, there are more than one book devoted to present the
concepts in detail
52/56
The Course
The primary goal of the course is to drive who has previous experience with
C/C++ and object-oriented programming to a proficiency level of (C++)
programming
• Proficiency: know what you are doing and the related implications
• Understand what problems/issues address a given language feature
• Learn engineering practices (e.g. code conventions, tools) and hardware/software
techniques (e.g. semantic, optimizations) that are not strictly related to C++
What the course is not:
• A theoretical course on programming
• A high-level concept description
What the course is:
• A practical course, prefer examples to long descriptions
• A “quite” advanced C++ programming language course
53/56
The Course
Organization:
• 22 lectures
• ∼1,500 slides
• C++03 / C++11 / C++14 / C++17 / C++20 / (C++23) / (C++26)
Roadmap:
• Review C concepts in C++ (built-in types, memory management, preprocessing,
etc.)
• Introduce object-oriented and template concepts
• Present how to organize the code and the main conventions
• C++ tool goals and usage (debugger, static analysis, etc.)
54/56
Who I Am
Federico Busato, Ph.D.
federico-busato.github.io
• Senior Software Engineer at Nvidia,
CUDA Mathematical Libraries
• Lead engineer of the Sparse Linear Algebra group
• Research/Work interests:
- Linear Algebra
- Graph Algorithms
- Parallel/High-Performance Computing
- Code Optimization
NOT a C++ expert/“guru”, still learning
55/56
A Little Bit about My Work
Our projects:
cuSPARSE GPU-accelerated sparse linear algebra library (matrix-matrix
multiplication, triangular solver, etc.), part of the CUDA Toolkit (8M
downloads every year)
cuSPARSELt Specialized library for sparse matrix-matrix multiplication that exploits the
most advanced GPU features such as Sparse Tensor Cores
NVPL Sparse CPU-accelerated (ARM) sparse linear algebra library
Top500 HPCG NVIDIA Supercomputing benchmark that performs a fixed number of
multigrid preconditioned (using a symmetric Gauss-Seidel smoother)
conjugate gradient (PCG) iterations
56/56
“What I cannot create,
I do not understand”
Richard P.
Feynman
56/56
“The only way to learn a new pro-
gramming language is by writing pro-
grams in it”
Dennis Ritchie
Creator of the C programming language
56/56
