How to build high-performance and resilient apps with Rust

08.09.2021•Raffael Schneider
Tech Rust Memory safety DevOps CLI Distributed Systems Concurrency How-to Tutorial

With Kotlin and Go and the implicit use of TypeScript we already have a multitude of modern programming languages in use. With Kotlin we have a language that perfectly complements the JVM stack with syntactic sugar and makes the code base meaningfully leaner. We wrote a brief introduction to this last year. There is also an introduction to Go that we set up when we started internally with Go development. There are now numerous applications that we have implemented completely in Go and are glad that we were able to incorporate Go as a skill, as the advantages are obvious during development. With this euphoria, we would also like to understand Rust better as a programming language and weigh up whether and for what this relatively new language would be best suited.

The starting point of our efforts to evaluate Rust is the consecutive top placement in the famous StackOver Developer Survey, which is published annually. Since 2016, Rust has been in first place in the category Most Loved Programming Language. We can’t just leave this to chance and have decided to take a close look at Rust and hope that we can also give you added value in terms of quality and sustainability.

Origins of Rust

Rust was originally developed as a personal project by Mozilla - employee Graydon Hoare. The well-known co-founder of Mozilla, inventor of JavaScript and today’s CEO of Brave Software, Brendan Eich also contributed to this. Rust has since been used extensively in Mozilla products. In addition to the general-purpose language C++, influences are clearly Haskell and Erlang, two functional programming languages which, when combined with a machine-level language such as C++, make a very interesting combination.

Meanwhile, many companies use Rust as a Systems Programming language, where memory allocation and security play a major role. There is a complete Unix-like operating system Redox, which is written in Rust and Microsoft also uses Rust to write new Windows components. There are also well-known big-tech players who include Rust in their stack, including Dropbox, Figma, Discord, 1Password and, last but not least, Facebook.

A restructuring caused by the COVID19 pandemic forced Mozilla to give the development of Rust and its ecosystem to a separate foundation. So there has been a dedicated Rust foundation since February 2021, which is financially supported by AWS, Huawei, Google, Microsoft and furthermore Mozilla in order to be able to guarantee its further development and continued existence.

Areas of application for Rust are broadly diversified as a General Purpose programming language according to the own documentation and include command-line tools, web services, DevOps tooling, embedded systems, audio and Video analysis, as well as transcoding, cryptocurrencies, bioinformatics, search engines, Internet-of-Things applications, machine learning and, last but not least, large parts of the Firefox web browser. It’s impressive how a relatively new language like Rust has already established itself in a wide variety of industries. We would now like to get to the bottom of this a little more and see what exactly distinguishes Rust.

Features of Rust

As mentioned at the beginning, Rust is a general purpose language and is suitable for all types of system programming. Rust lays the main features on the balance between performance and system stability. The compiler plays a very central role here, because it is opinionated, quite similar to Go, so he knows what to accept and what not. When building a codebase, compilers usually check whether the code is valid due to the syntax and types (at typed langauges) and leaves a whole class of potential errors to the runtime.

With conventional languages such as Java, C++ or Go, on the other hand, in addition to the actual application logic, a garbage collector also runs at runtime, which automatically operates memory management and, in simple terms, releases memory units that have become obsolete for reuse. Rust has no garbage collection and checks at compile time with a so-called escape analysis whether the code could not lead to any memory-induced errors. This means that Rust can build leaner and more efficient artifacts by omitting the garbage collector. As compensation for garbage collecting, Rust offers an understanding of ownership, which a budding Rust programmer must first acquire. We’ll explain this ownership concept later in this post.

Before we get into the features of Rust, let’s first hear how the official Rust documentation sums up their own language:

Rust is for people who crave speed and stability in a language. By speed, we mean the speed of the programs that you can create with Rust and the speed at which Rust lets you write them. The Rust compiler’s checks ensure stability through feature additions and refactoring. This is in contrast to the brittle legacy code in languages without these checks, which developers are often afraid to modify. By striving for zero-cost abstractions, higher-level features that compile to lower-level code as fast as code written manually, Rust endeavors to make safe code be fast code as well.

The Rust language hopes to support many other users as well; those mentioned here are merely some of the biggest stakeholders. Overall, Rust’s greatest ambition is to eliminate the trade-offs that programmers have accepted for decades by providing safety and productivity, speed and ergonomics.

The Rust Programming Language Documentation, Introduction [source]

One notices that the main problem in the development of Rust was how to achieve the desired programming comfort in the form of abstraction and high-level properties with a focus on machine-level performance; the so-called zero-cost abstractions.

To briefly summarize the features and benefits of Rust, you can consider the following list:

  • Consistent memory security guaranteed by the opinionated Rust compiler

  • Explicit concurrency and parallelism through Data Ownership Model

  • Abstraction and development convenience for free (no performance drawbacks)

Well, now we have roughly described what makes Rust stand out. For techies who have never had to deal with machine-related concepts, one or the other concept may be completely new and probably don’t know exactly what to do with it in the first step. For this reason, we should first look at the most important thing about Rust: the data ownership model. Hopefully this will make everything a little more understandable.

Ownership, Borrow Checking and Escape Analysis

As already announced, the greatest feature of Rust is the absence of garbage collecting and the associated concept of Ownership, but in this context it is best translated as utilization. The concept of ownership defines the life cycle of data during the runtime of a Rust program.

The Rust documentation defines 3 rules for the concept of ownership. These rules are checked by the Borrow Checker, a feature of the Rust compiler, during the build time to ensure that the life cycle of a data unit is valid. The Borrow Checker uses a pattern known in compiler construction with the name Escape Analysis. Escape Analysis tracks the runtime of a variable until it falls out of the scope of application (scope) and decides on the basis of this what has to be done with the freed memory allocation. JVM languages such as Java or general garbage-collected languages do this at runtime via dynamic allocation of variables. Back to the 3 rules. The rules are as follows:

1 . Each value in Rust has a variable that’s called its owner. Each value has a variable in Rust, and this variable is its owner.

2 . There can only be one owner at a time. There can only ever be one owner of a value at a given point in time.

3 . When the owner goes out of scope, the value will be dropped. If the owner leaves the scope of the value, the value is deleted.

It makes sense to briefly recall the difference between heap and stack from the machine-level memory management. Heap denotes dynamic memory, which can freely release contiguous memory sections at runtime as soon as they are marked for recycling. The Stack, on the other hand, is a static stack memory which accepts and outputs data values according to the last-in-first-out principle, or LIFO for short. At runtime, data values are typically recorded in the heap if its life cycle, i.e. the potential mutability (will the values change or not) is not known in advance. More precisely, a compiler, like in C++, checks the scope of a variable at build time and defines whether this value has a short, previously known lifespan or not, and on the basis of this whether the value is added to the heap or the stack.

Here we have an example (borrowed from dev-notes.eu) that illustrates this in Rust code.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

fn main() -> Result<(), &'static str> {
    let mut a = [1,2,3,4];
    println!("{:?}", a); // Line 1 output
    
    {
        let b = &mut a[0..2];
        // You can't access a at this point because it has been mutably borrowed
        // from. The following line won't compile, with the error message:
        // `cannot borrow `a` as immutable because it is also borrowed as mutable`:
        // println!("a: {:?}", a);

        println!("b: {:?}", b); // Line 2 output
        b[0] = 42;
        println!("b: {:?}", b); // lIne 3 output
    }

    // The borrow is scope-dependent - a is accessible again because the 
    // variable that borrowed it is out of scope:
    println!("a: {:?}", a); // Line 4 output
    Ok(())
}

Output:

1
2
3
4

[1, 2, 3, 4]
b: [1, 2]
b: [42, 2]
a: [42, 2, 3, 4]

Thus one can say that there can always be two types of referencing of values:

  • Shared references with &

  • Mutable, variable references with &mut

Furthermore, a reference to a value can never exist longer than the value that is referenced. The idea of ownership goes a little further at Rust and makes the first steps in the language a little more complex, since the management of data lifecycles is probably abstracted in other, more well-known languages such as Java or Python and is not part of the programming tasks. You just have to understand that Rust creates the possibility to write correct code, which focuses on security and safety with one main characteristic, which makes its artifacts - provided you know what you are doing - incredibly robust and stable. If you want to learn more about ownership and how to deal with data lifecycles at Rust, I can recommend the official documentation:

https://doc.rust-lang.org/book/ch04-01-what-is-ownership.html

Installation with RustUp

To install Rust there is a toolchain called rustup. In a Unix-like environment such as GNU / Linux or macOS, you can pull down rustup directly using the following command and thus initiate the installation process:

1

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

ℹ️ With Windows machines it is sufficient to download rustup-init.exe and run it. More information on this under https://forge.rust-lang.org/infra/other-installation-methods.html

In addition to rustup, the actual toolchain administration, the executed shell script also installs the following elements:

  • rustc: The actual Rust compiler which generates the binaries from Rust source code.

  • rustfmt: A format binary which, similar to go fmt, automatically formats the source code accordingly.

  • cargo: Cargo is the build system and package manager from Rust. We will use it to build our Rust projects and resolve any dependencies.

  • std: Finally the standard libs of the language are also installed.

Hello b-nova!

A ‘Hello b-nova!’ Function is written relatively quickly in Rust. You can do your first steps without having to install Rust on the local machine in the official Rust Playground.

1
2
3

fn main() {
    println!("Hello b-nova!");
}

Simply open the playground, copy the code snippet and click Run! You can clearly see the output of the build process, as well as the actual output when it is executed.

1
2
3
4
5
6
7

--- Standard Error ---
   Compiling playground v0.0.1 (/playground)
    Finished dev [unoptimized + debuginfo] target(s) in 1.27s
     Running `target/debug/playground`
     
--- Standard Output ---
Hello b-nova!

If you are interested, you can start the official tutorial in the Playground, the so-called tour of Rust. This has the same structure and goes through the entire language specification and learns how to write if / else and for loops with Rust, what the individual data types are, etc. Perhaps the best thing to do is take a look at table of content of the tutorial, which incidentally is completely translated into German.

Building with Cargo

As soon as we set up a real Rust project, cargo is an important element in the entire life cycle of the application to be built. Make sure that Cargo is correctly installed and that the Rust version is up-to-date.

1
2
3

❯ cargo --version && rustc --version
cargo 1.54.0 (5ae8d74b3 2021-06-22)
rustc 1.54.0 (a178d0322 2021-07-26)

With $ cargo new hello-bnova Cargo is informed that we would like to have a project with the name hello-bnova. Cargo takes over and provides us with a complete directory.

1
2

❯ cargo new hello-bnova
     Created binary (application) `hello-bnova` package

Cargo not only creates the directory, but also provides a canonical project structure which, in addition to .git and a .gitignore, also determines the src/ \ folder, which contains a main.rs, as well as Cargo.toml and Cargo.lock files. Very useful if you ask me.

1
2
3
4
5
6
7
8
9

❯ tree -a
.
├── .git
│   └── ...
├── .gitignore
├── Cargo.lock
├── Cargo.toml
└── src
    └── main.rs

Before we open the main.rs, we will briefly look over the Cargo.toml and explain what it is all about.

Cargo.toml

1
2
3
4
5
6
7
8

[package]
name = "hello-bnova"
version = "0.1.0"
edition = "2018"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

As already suspected, this toml file is a meta file of the Rust application. Using this file, Cargo resolves vulnerable dependencies and knows what the application is like. Here we will later maintain a dependency and see how Rust does it.

The src / main.rs contains the same snippet that we ran earlier on the playground.

main.rs

1
2
3

fn main() {
    println!("Hello, world!");
}

With the Cargo CLI we can now build the project we just created. Normally this works with cargo build, but we can compile and run its artifact on a single line.

1
2
3
4

❯ cargo run
    Finished dev [unoptimized + debuginfo] target(s) in 0.00s
     Running `/Users/rschneider/Development/rust/hello-bnova/target/debug/hello-bnova`
Hello, world!

Now we want to play around a little and will use the popular CLI framework Clap. First, let’s add the dependency https://crates.io/crates/clap. At the moment of the TechUp this is available in the version 2.33.3.

Cargo.toml

1
2
3

...
[dependencies]
clap = "2.33.3"

In the main.rs we build the basic structure of Clap right away.

main.rs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65

// (Full example with detailed comments in examples/01b_quick_example.rs)
//
// This example demonstrates clap's full 'builder pattern' style of creating arguments which is
// more verbose, but allows easier editing, and at times more advanced options, or the possibility
// to generate arguments dynamically.
extern crate clap;
use clap::{Arg, App, SubCommand};

fn main() {
    let matches = App::new("Hello b-nova!")
                          .version("1.0")
                          .author("Raffael Schneider <raffael.schneider@b-nova.com>")
                          .about("Says Hello b-nova!")
                          .arg(Arg::with_name("config")
                               .short("c")
                               .long("config")
                               .value_name("FILE")
                               .help("Sets a custom config file")
                               .takes_value(true))
                          .arg(Arg::with_name("INPUT")
                               .help("Sets the input file to use")
                               .required(true)
                               .index(1))
                          .arg(Arg::with_name("v")
                               .short("v")
                               .multiple(true)
                               .help("Sets the level of verbosity"))
                          .subcommand(SubCommand::with_name("test")
                                      .about("controls testing features")
                                      .version("1.3")
                                      .author("Someone E. <someone_else@other.com>")
                                      .arg(Arg::with_name("debug")
                                          .short("d")
                                          .help("print debug information verbosely")))
                          .get_matches();

    // Gets a value for config if supplied by user, or defaults to "default.conf"
    let config = matches.value_of("config").unwrap_or("default.conf");
    println!("Value for config: {}", config);

    // Calling .unwrap() is safe here because "INPUT" is required (if "INPUT" wasn't
    // required we could have used an 'if let' to conditionally get the value)
    println!("Using input file: {}", matches.value_of("INPUT").unwrap());

    // Vary the output based on how many times the user used the "verbose" flag
    // (i.e. 'myprog -v -v -v' or 'myprog -vvv' vs 'myprog -v'
    match matches.occurrences_of("v") {
        0 => println!("No verbose info"),
        1 => println!("Some verbose info"),
        2 => println!("Tons of verbose info"),
        3 | _ => println!("Don't be crazy"),
    }

    // You can handle information about subcommands by requesting their matches by name
    // (as below), requesting just the name used, or both at the same time
    if let Some(matches) = matches.subcommand_matches("test") {
        if matches.is_present("debug") {
            println!("Printing debug info...");
        } else {
            println!("Printing normally...");
        }
    }

    // more program logic goes here...
}

Let’s build the project again with cargo build.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

❯ cargo build
    Updating crates.io index
  Downloaded textwrap v0.11.0
  Downloaded unicode-width v0.1.8
  Downloaded vec_map v0.8.2
  Downloaded clap v2.33.3
  Downloaded strsim v0.8.0
  Downloaded atty v0.2.14
  Downloaded ansi_term v0.11.0
  Downloaded 7 crates (282.3 KB) in 0.50s
   Compiling libc v0.2.99
   Compiling unicode-width v0.1.8
   Compiling strsim v0.8.0
   Compiling bitflags v1.3.2
   Compiling vec_map v0.8.2
   Compiling ansi_term v0.11.0
   Compiling textwrap v0.11.0
   Compiling atty v0.2.14
   Compiling clap v2.33.3
   Compiling hello-bnova v0.1.0 (/Users/rschneider/Development/rust/hello-bnova)
    Finished dev [unoptimized + debuginfo] target(s) in 6.98s

Now we should have a binary hello-bnova in our target/ \ directory, which we can run as a CLI app.

1
2

❯ ./target/debug/hello-bnova -V
Hello b-nova! 1.0

The basic structure of Clap does a little more than just print ‘Hello bnova!’, but the CLI app already works in principle. Congratulations! 🍀

Also, worth mentioning are the incredibly expressive and understandable error messages during compilation. I tried to replace “ with ’ in a string expression. The compiler does not show me the line, but suggests several solutions, which I can often simply copy from the error message.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

❯ cargo build
   Compiling hello-bnova v0.1.0 (/Users/rschneider/Development/rust/hello-bnova)
error: character literal may only contain one codepoint
  --> src/main.rs:46:27
   |
46 |         3 | _ => println!('Don't be crazy')
   |                           ^^^^^
   |
help: if you meant to write a `str` literal, use double quotes
   |
46 |         3 | _ => println!("Don"t be crazy')
   |                           ^^^^^

error[E0762]: unterminated character literal
  --> src/main.rs:46:42
   |
46 |         3 | _ => println!('Don't be crazy')
   |                                          ^^

error: aborting due to 2 previous errors

For more information about this error, try `rustc --explain E0762`.
error: could not compile `hello-bnova`

To learn more, run the command again with --verbose.

In this example, the footprint of a CLI app with Rust is 3.9 MB, which is certainly half the size of a counterpart in Go (with the CLI-framework Cobra).

1
2

❯ du -h target/debug/hello-bnova
3.9M	target/debug/hello-bnova

A few more words about Cargo and Crates

The Rust ecosystem seems to have matured after a good ten years of existence and currently offers libraries for all possible use cases. The in-house package manager Cargo has its own crate registry, namely https://crates.io/. On it, you can find all the existing crates that can be built into a Rust project.

At this point we would like to briefly provide a brief overview of which crate could cover which use case. Actually, each of the entries below should have its own TechUp, as they are extensive in their respective competencies. But we will summarize them briefly here.

  • Clap: Clap is a CLI framework with which you can program a CLI app, just like with Cobra at Go.

  • Rayon: With Rayon you bring parallelism and concurrency into a project.

  • Tokyo: With Tokio you build event-driven, highly available, asynchronous applications. Tokyo is specially designed to be lightweight.

  • Rocket: Rocket is the to-go web framework for Rust and offers the usual features that one can expect from a web framework.

  • WASM: wasm-pack is a WebAssembly BuildTool that can generate wasm artifacts from Rust. Since the introduction of WebAssembly in 2019, every web browser supports the possibility of executing wasm machine code instead of using an ECMAScript-compatible language for client-side applications. Besides Kotlin and Go, Rust is one of the few languages that support WASM to date. Rust has since established itself as the standard for WASM applications.

Conclusion

Rust is a modern language that approaches things a little differently and is the ideal candidate for high-performance and stable system development. Even if many people are not familiar with Rust and its use still seems to concentrate on marginal niche areas, Rust has interesting properties that make Rust the language of choice in a wide range of applications, not least robust web Services and CLI apps.

At b-nova, we specialize in testing new technologies for their applicability and sustainability and, if our prospects for quality increase, also transfer them to the use and success of customer projects. We are sure that Rust will establish itself as another valuable tool in our toolbox and we are already looking forward to demonstrating Rust in a project for you. Stay tuned!

https://www.rust-lang.org/

https://github.com/rust-unofficial/awesome-rust

https://serokell.io/blog/rust-guide

https://tourofrust.com/

https://doc.rust-lang.org/stable/rust-by-example/

This text was automatically translated with our golang markdown translator.